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I. INTRODUCTION 


A. BACKGROUND 

Intuitively the proposal seemed economically beneficial to the Naval Reserve: take 
an information system costing over three million dollars per year to operate on a 
mainframe computer and move the system to a network of minicomputers costing a few 
hundred thousand dollars to purchase and the same each year to operate. The savings 
should be substantial and system support might even improve, since the operators of the 
mainframe had not been very responsive to user requests lately. This was the proposal 
presented to me when Commander Naval Reserve Force (COMNAVRESFOR or 
CNRF)! staff members asked, in July 1991, for an economic analysis of the redesign 
alternatives for the Reserve Financial Management System (RESFMS). As it turns out, 
more than a year later, the outcome of the analysis produced results similar to those 
which intuition suggested. However, the process of evaluation uncovered questions and 
raised issues not originally considered which are of great import to the success of any 
redesign effort. Also, no matter how strongly managers are convinced that an 
information system development project would be beneficial, if that judgement is not 


based in objective analysis, funds for development will not be approved in the 


'Members of the Naval Reserve generally refer to Commander Naval Reserve Force 
as COMNAVRESFOR. For purposes of brevity in correspondence or references it is 
sometimes shortened to the four letters CNRF. This usage will be preserved such that 
the body of the text will use COMNAVRESFOR, and tables, graphs, and references will 
generally use CNRF. 


Department of Defense (DoD). Therefore, a formal analysis of alternatives was not only 


instructive to the planning process, but required by DoD policy. 


B. OBJECTIVES 

The primary objective of the study was to ascertain the economic benefits, if any, 
to the Naval Reserve of redesigning RESFMS to operate in a hardware environment 
other than the mainframe on which it runs at present. This analysis was to be done in 
a manner consistent with current DoD directives and following established economic 
analysis principles. 

The target hardware configuration for the redesign was determined by 
COMNAVRESFOR personnel and presented to me for use in the economic analysis. 
Yet, although the type of computer to be used was determined, the number of 
minicomputers required to run RESFMS was not determined prior to my evaluation. 
Therefore, one of the first objectives of this study was to validate the adequacy of the 
computer chosen to effectively operate RESFMS, and to determine the number of 
computers required for this purpose. 

The next objective of the study was to determine all current system costs and 
benefits and to estimate alternative system costs and benefits. Determination of current 
system costs was fairly straightforward. However, estimation of alternative system costs 
turned out to be the most difficult part of this entire study. The determination of 
hardware costs was relatively simple. The estimation of the cost of developing new 


system software, on the other hand, was a daunting problem which caused further study 


and analysis of the problem of software cost estimation in general, and of the problem 
in the Department of Defense, in particular. 

The final objective of this study was to analyze the costs and benefits identified in 
such a way that decision makers could use the analysis as a basis for program 
development approval or disapproval. In keeping with this objective, an analysis has 
been made using several economic analysis tools and risk analysis procedures applied to 
the results. Consequently, the final outcome of the study provides not single values, but 


ranges of values and probabilities of outcomes for decision maker evaluation. 


C. SCOPE, LIMITATIONS, AND ASSUMPTIONS 

This study is limited to the alternatives and data provided by COMNAVRESFOR 
for analysis. General principles and procedures examined, such as economic analysis, 
software development cost estimation, and risk analysis, will be discussed as they relate 
to information systems development in the Department of Defense. Assumptions related 
to specific items of the analysis will be discussed when those items are described and 


evaluated. 


D. ORGANIZATION OF STUDY 

One reason why an economic analysis of redesign alternatives for RESFMS is 
being performed is because such an analysis is required within DoD. To understand 
what is required and why, it is helpful to examine the background of economic analysis 


in general as well as the history and current requirements for economic analysis in DoD. 


These issues, along with a description of assumptions for this study that result from 
current DoD directives, will be discussed first. 

In order to understand the rationale for a proposed redesign of RESFMS, the 
history of its development and evolution should be understood. Knowledge of the current 
configuration is essential to understanding current operating costs and benefits; and the 
target configuration and rationale used to determine it are useful in understanding the 
potential costs and benefits of the redesign alternative. Therefore, a discussion of these 
aspects of RESFMS will be next. 

Since the process of economic analysis is central to this study, and the selection of 
economic analysis tools critical to the results obtained, the structure of economic 
analysis, definitions of major terms, and description of appropriate tools will be 
examined. The economic analysis tools selected for this study will be listed and briefly 
explained. 

The problem of estimating the cost of software development for RESFMS was the 
most significant problem encountered in this study. Therefore, a chapter will be devoted 
to an explanation of the nature of the software cost estimation problem in general, some 
of the methods and tools available to settee estimates, and the applicability of this 
problem to information systems development in DoD. 

Next, a detailed description of costs chosen for this analysis is presented. Special 
attention is given to the source data, reasoning, assumptions and methods used to 


determine software development cost estimates in the case of RESFMS. 


Once the tools have been chosen and all costs identified and quantified, the analysis 
is performed. The first computations of economic analysis were done using actual costs, 
if known, and expected values if cost was uncertain. Then, risk analysis is performed 
using the full range of possible values for uncertain cost estimates. The results of risk 
analysis show a range of possible outcomes and the probability of obtaining those values. 

Finally, a recommendation will be made, based on the results of economic analysis, 
as to what alternative will have the highest probability of positive financial benefit for 


the Naval Reserve. 





While, economic analyses provide guidelines tor making decisions, some person or group 


must ultimately accept the risk and make the decisions. 


B. INFORMATION SYSTEM EXPENDITURES 
In the last three decades, rapid advancement in information systems (IT) technology 
has resulted in major investment in IT hardware, software, and related systems by both 
large corporations and very small firms (Cash and others, 1988). An indication of the 
scope of these investments and investing trends can be obtained by looking at capital 
investment in high-technology industries. 
It is clear that United States capital investment is increasingly turning towards high- 
technology industries. ... Capital expenditures for basic industrial equipment have 
been reduced from 25% of all capital spending in the 1960s to present levels of less 
than 13% of all capital spending. Meanwhile, spending for high-tech equipment 
rose from 12% in the 1960s to present levels of more than 30% of all capital 
spending. (Strassmann, 1985) 
The need to analyze these growing expenditures and informed decisions has caused 


organizations to apply economic analysis methods to ADPE acquisition and information 


system development. 


C. GOVERNMENT REGULATIONS CONCERNING ADPE 

The Congress of the United States, recognizing the importance of capital 
expenditure on ADPE, has enacted legislation and given policy direction through hearings 
and reports. In October 1965 Congress enacted Public Law 89-306, known as the 
Brooks Act, establishing the basic policy for the management of data processing 


equipment in the Federal Government. Public Law 99-500, known as the Paperwork 


Reduction Reauthorization Act of 1986, expanded the scope of the Brooks Act "to 
include telecommunications resources, software, and computer-related services such as 
computer service bureaus and contract programming." (GSA Overview Guide, 1990) 

The Brooks Act charges the Office of Management and Budget (OMB) with 
developing management policy and providing fiscal control of ADPE. In its circular 
entitled "Management of Federal Information Resources", the general principles of 
ADPE acquisition and development are stated. One of the principles cited states: 

In order to minimize the cost and maximize the usefulness of government 
information activities, the expected public and private benefits derived from 
government information, insofar as they are calculable, should exceed the public 
and private costs of the information. (OMB Circular A-130, 1985) 
To determine if benefits derived do, in fact, exceed the costs, one must use some form 
of economic analysis. Procedures and policies for economic analysis of ADPE in the 
government are delineated by the General Services Administration (GSA). 

According to the Brooks Act, GSA is to "coordinate and provide for the economic 
and efficient purchase, lease, and maintenance of automated data processing equipment 
by Federal agencies." Although GSA is given exclusive authority to procure ADPE 
resources, it is also granted the power to delegate authority for procurement to Federal 
agencies as necessary. (GSA Overview Guide, 1990) 

The primary document containing GSA regulations for automated data processing 
equipment is the Federal Information Resources Management Regulation (FIRMR). In 


the FIRMR, automated data processing equipment and associated information systems are 


referred to as Federal Information Processing (FIP) resources. In its section on 


acquisition, the FIRMR directs that an analysis of alternatives be made prior to acquiring 
or developing any system and that "in the analysis of alternatives, agencies shall calculate 
the total estimated cost, using the present value of money, for each feasible alternative 
unless the cost of the acquisition is $50,000 or less." (FIRMR, 1990) It also directs, 
when calculating the cost of each alternative, agencies must follow guidance in OMB 
Circular No. A-94, "Discount Rates to be Used in Evaluating Time-Distributed Costs and 
Benefits." The FIRMR, further, specifically lists those costs to be included and those 
to be excluded in any analysis. 

Using its power to delegate, GSA has set criteria to determine which acquisitions 
and development efforts must be reviewed and approved by GSA itself, and which may 
be approved by the government agency acquiring the system. Each agency, in turn, has 
produced its own set of regulations and directives governing FIP resource acquisition 


within that agency. 


D. FIP RESOURCE ACQUISITION AND MANAGEMENT IN DOD 
The armed services are the heaviest user of FIP resources in the U.S. Government 


(Kellner, 1991), spending almost nine billion dollars on automatic data processing in 


OMB Circular A-94 specifically states: "This Circular would not apply to the 
evaluation of decisions concerning how to select automatic data processing equipment, 
guidance for which is OMB Circular No. A-54 and OMB Bulletin No. 60-6." Yet, 
GSA, in the FIRMR, §201-20.203-1 (c), directs: "Agencies shall follow guidance in 
OMB Circular No. A-94, ’Discount Rates to be Used in Evaluating Time-Distributed 
Costs and Benefits,’ when calculating the cost of each alternative." Since the FIRMR 
is the most recent of the documents (1990 vice 1972), it takes precedence and procedures 
in OMB Circular A-94 are to be followed in spite of the disclaimer. 


1990 alone (HASC, 1989). To manage these resources, the Department of Defense 
(DoD) has established its own procedures and regulations regarding FIP resources. DoD 
Directive 7920.1, “Life-Cycle Management (LCM) of Automated Information Systems 
(AISs)," DoD Directive 7920.2, "Automated Information System (AIS) Life-Cycle 
Management Review and Milestone Approval Procedures," and DoD Directive 5000.1, 
"Defense Acquisition," all require that regulations found in the FIRMR be followed in 
FIP resource acquisition. As noted earlier, the FIRMR requires an economic analysis 
of alternatives. Within DoD, procedures to be followed in this economic analysis are 
found in DoD Directive 7041.3, “Economic Analysis and Program Evaluation for 
Resource Management." Individual services, including the Department of the Navy 
(DoN) have written their own directives, following OMB, GSA, and DoD guidance, 
which further spell out procedures to be followed in each service. The general principles 
of economic analysis found in methods used in the private sector are found in these 
directives as well. Thus, careful consideration 1s to be given to the time value of money 
and the effects of interest rates and inflation. 

In spite of this plethora of direction concerning automatic data processing systems, 
DoD has experienced significant aroulene analyzing, acquiring, developing and 
managing these systems. A series of reports from both houses of Congress between 1988 
and 1990 documented these problems. Responding to GAO reports of mismanagement 
of ADPE in DoD, the House Armed Services Committee (HASC) recommended, in July 
1989, that the services’ automatic data processing request be reduced by $165.5 million. 


The HASC also stipulated a requirement that DoD develop a plan of action by February 


1, 1990 on how to resolve the identified problems. The implication was that further 


reductions would result if the deadline were not met. 


E. DOD CORPORATE INFORMATION MANAGEMENT INITIATIVE 

In response to the above HASC requirements, Deputy Secretary of Defense Donald 
Atwood announced, in October 1989, a Corporate Information Management (CIM) 
initiative. This initiative involves radical changes in the way information resources are 
managed in DoD. A thorough discussion of the tenets and implications of CIM is 
beyond the scope of this thesis. However, it is important to understand the basic premise 
of CIM and its effects on automatic data processing system development. 

Central to the philosophy and method of CIM is the concept that "it 1s not about 
technology; it is about business processes and managing information." (Brewin, 1991) 
The theory is that businesses "gain strategic advantage by changing the way they work, 
not by automating old or inefficient methods." (Brewin, 1991) Thus, CIM seeks to have 
all DoD agencies analyze their basic business processes. Once a business process is 
understood, it can be redesigned with the goal of achieving the greatest efficiency 
possible in every business activity. Then, and only then, is information technology 
considered as a means of implementing this process. 

By supporting functional managers in streamlining business methods, DoD’s 
corporate information management initiative will aid the Department in achieving 
the aggressive savings targets established by the Defense Management Report. To 


achieve the highest savings, CIM investments must be based on a functional 
economic analysis of business activities or operations. (DDI Memo, 1991) 


The functional economic analysis now required for all CIM, and thus ADPE, 
investment decisions has been spelled out in memoranda and training presentations to 
DoD management personnel. The functional economic analysis, also called a Business 
Case, follows and amplifies upon policy and procedures contained in DoD Directive 
7041.3 described above. One part of a Business Case is an analysis of alternatives for 
information systems to support a redesigned business process. This analysis of 
alternatives 1s to follow the guidelines set forth in the FIRMR, and in OMB, DoD, and 
Department of the Navy (DoN) directives except as amended by CIM. The chief effects 
of CIM on this part of the analysis are an increased emphasis on the financial impacts 
of risk, and the requirement to express potential benefits in cash terms. Specific 
procedures will be discussed in Chapter IV. 

The directives on guidelines discussed so far, all apply to all agencies within DoD. 
The Naval Reserve, as part of the Department of the Navy (DON), is a DoD agency. 
Commander Naval Reserve Force (COMNAVRESFOR) is responsible for operating and 
developing several information systems. One of those systems is the Reserve Financial 
Management Systems (RESFMS), which has been proposed as a candidate for redesign. 
Project approval and allocation of funds for the redesign of RESFMS will be contingent 
upon the results of a Business Case presented by COMNAVRESFOR to higher approval 


authority. 


F. RESFMS BUSINESS CASE ASSUMPTIONS 

The purpose of this thesis is to perform an economic analysis of the alternatives for 
the Reserve Financial Management System (RESFMS). Such an economic analysis will 
be part of the Business Case presented by COMNAVRESFOR to gain approval for 
proceeding with the redesign effort. The other usual component of a Business Case is 
an analysis of the business processes of the agency making the proposal. An analysis and 
redesign of the business process of the Naval Reserve is beyond the scope of this thesis. 
I assume that such an analysis, if required, has already been done and that the functional 
requirements of RESFMS, generated by COMNAVRESFOR, support efficient business 
processes of the Naval Reserve. I further assume that a decision has already been made 
that an information system, in the form of RESFMS, provides the best means to perform 
the functional reauireniesits given. 

This analysis, then, will meet the requirements for an analysis of alternatives for 
an information system that may be a part of a Business Case to be produced by the Naval 
Reserve and used as a decision tool when considering future budget and development 


plans. 


II. RESERVE FINANCIAL MANAGEMENT SYSTEM (RESFMS) 


A. DESCRIPTION OF RESFMS 

The Reserve Financial Management System (RESFMS) is an information system 
used by the Naval Reserve to manage the Reserve Personnel Navy (RPN) appropriation 
account, to issue active duty orders to reservists, and to arrange for travel for reservists 
in conjunction with both active duty and inactive duty training orders. As such, the 
system crosses two major Department of Defense (DoD) functional areas: Manpower, 


Personnel and Training (MPT), and Financial Management (FM). 


B. HISTORY OF RESFMS 

In the 1970s, the Naval Reserve operated an information system for issuing active 
duty orders, called Order Writing, and a system for accounting, called RPN Accounting, 
as two distinct systems operating on separate mainframe computers. There was no 
interface between these systems. Information from one system was manually transferred 
to the other. In 1979 the Naval Reserve experienced an over-obligation of the RPN 
account due to poor management of active duty order issue and travel expenses. An 
over-obligation of a Congressional appropriation is prohibited by law under Title 31 
United States Code 1517, and may incur serious consequences for the person responsible 
such as suspension without pay, removal from office, fines, or imprisonment (Practical 


Comptrollership, 1992). Asa result, Congress mandated in 1980 that the Naval Reserve 


would develop an information system to correct the accounting and order writing 
problems, that no more over-obligations would occur, and that the new system must be 
operational by 1984 (Lacy, 1992). 

Staff members of COMNAVRESFOR assigned to Code 10, the computer systems 
management division, decided the new system would have three integrated subsystems: 
Active Duty Order Writing, Travel, and RPN Accounting. They decided on an 
incremental development approach. Navy Regional Data Automation Center 
(NARDAC), New Orleans was contracted to develop and run the system on their Sperry 
1100/90 mainframe computer. They wrote the programs in COBOL and used a 
proprietary hierarchical database provided by Sperry called DMS1100. The Active Duty 
Order Writing module was first operational in February 1983, and the Travel module in 
April 1984. In 1983 NARDAC informed the Naval Reserve that they would be unable 
to produce the entire system and meet the 1984 deadline. NARDAC suggested that a 
civilian contractor be hired to do the RPN Accounting module. Therefore, the Naval 
Reserve contracted with CACI, Inc. who subcontracted with SYSCON, Inc. to produce 
the RPN Accounting module. RPN Accounting was first operational in October 1984. 
Upon system completion, SYSCON, Inc. was contracted to provide software maintenance 
and NARDAC provided hardware maintenance and support. Initial design, development, 
and implementation of the system cost nine million dollars. Since 1984, the company 
providing contract software maintenance has changed twice. Total system costs through 
1990 were in excess of $50 million including both investment and operating expense 


(Blaylock, 1990). 


a) 


In January 1987 RESFMS became the first Navy information system to be certified 
as compliant with requirements prescribed tor federal agencies by the Office of 
Management and Budget (OMB), General Accounting Office (GAO), Department of the 
Treasury, Department of Defense (DoD), and Department of the Navy (DON). Very few 


of the 121 Navy accounting systems have achieved this certification. 


C. EVOLUTION OF RESFMS 

Since its inception, seven major additional functions, and four major system 
interfaces have been added to RESFMS (Lacy, 12 August 1991). Many small features 
have been added as well. These additions will be discussed in the following section on 
current configuration. Considerable effort has also been directed toward software 
maintenance which has modified the structure and size of many programs considerably. 
This aioe effort has been needed both because of changes in the operating 
environment and because of the methods and procedures which were used in initial 


software development. 


1. Changes in Operating Environment 
In May 1986, Burroughs acquired Sperry (Barbetta, 1986). The merged 
company, called Unisys, offered a hardware upgrade to a new machine, the Unisys 
1100/92. This upgrade was installed in NARDAC New Orleans in the late 1980s. The 
92 is similar to the 90 but has two CPUs instead of one. The two processors can operate 
as a tightly coupled pair, essentially doubling the computing power of 7.5 MIPS to a 


rating of 15 MIPS. The processors can also be de-coupled in software and operate as 
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two separate computers sharing the same peripherals. Minor changes in code were 
required to run RESFMS on the upgraded machine. 

More important to the maintenance effort was the fact that a number of 
additional functions, features and interfaces were added to the system between 1984 and 
1991. For example, additional functions included order generation for Health Sciences 
Education Command (HSTEC), issuance of travel claim vouchers, and calculations using 
the Uniform Chart of Accounts (UCA). Features added included Three Minute Orders 
and batch printing of active duty orders (Lacy, 12 August 1991). New interfaces were 
also added with Micro Claims Processing System (MCPS), Integrated Disbursing and 
Accounting Financial Management System (IDAFMS), Centralized 
Expenditures/Reimbursement Processing System (CERPS), and Navy Standard Claimancy 
Accounting Module (NSCAM) (CNRF RESFMS Briefing Notes, June 1991). These 
additions required numerous software patches. Since they were not part of the original 


design, integration and debugging were very difficult. 
2. Problems of Initial Software Development 


a. Analysis and Design 
Inadequate requirements analysis and product design were performed 
before coding initially began in 1981. NARDAC adopted a "code and fix" approach to 
development, believing that there was insufficient time to do a thorough job of 
requirements analysis and design prior to coding. One result of the lack of adequate 


design is that more errors are included in the code. If coding begins without a clear idea 


of where the project is headed and exactly how to get there (the result of detailed 
design), many false starts are made on segments of program code, not all of which are 
removed during debugging. Even the debugging process suffers from poor design. 
Without a detailed product design, detailed test plans cannot be generated that fully test 
all program segments and functions. The result is that the system still includes many 
undiscovered errors even after it is made operational and those errors may be difficult 
to find because of the poor structure and design. Such is the case with RESFMS (Lacy, 
30 June 1992). Therefore, the Active Duty Order Writing and Travel modules have been 


very difficult to maintain. 


b. Redundant Code 

Since coding was begun without a clear idea of overall product design, 
procedures that should have been identified as common to many parts of the application 
were not. “Ke a result, when sections of the program were encountered with similar 
function to those previously coded, large sections of code were copied and slightly 
modified to fit the new situation. Modern programming practice and structured design 
principles would have these common procedures located in a single common module 
using changing input parameters to produce the variations of output (Pressman, 1992). 
Thus, if system maintenance dictates that the procedure needs to be modified, it can be 
easily found and changed in one location which results in the required modification to 
the entire system. In RESFMS the opposite is true. In order to change one particular 
function, all instances of a segment throughout the application must be found and 


changed. This has created numerous problems for maintenance programmers. 
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c. Database Design 
The database was not well designed. It was not adequately normalized, 
was created piecemeal, contained redundant elements with different names, and contained 


data never used by the application. 


d. Global Variables 
Global variables were also common. With many different modules 
acting on the same common variables, maintenance programmers often found that small 


changes in one module had unplanned and unwanted effects throughout the system. 


3. The Maintenance Challenge 
When Systems Engineering and Management Associates, Inc. (SEMA) took 
Over as software maintenance contractor for RESFMS in 1989, they inherited a system 
that had been poorly designed and coded with little structure, redundant code, redundant 
data elements, little structure, and prone to errors. Because of both poor programming 
practices and addition of functions and interfaces, by 1989 RESFMS had grown in size 


to include (Lacy, 12 August 1991): 


@ Over two million lines of COBOL code 


4,500 COBOL programs 


@ 250 record types in database 


15.5 million records in database 


Even though the system was fully functional and had met stringent audit standards, the 
continued discovery of errors and the addition of new functions and interfaces made it 


a maintenance nightmare. 


4. The Maintenance Solution 

The COMNAVRESFOR program manager, Ms. Coreen Lacy, mandated that 
for the first eight months of the new contract no software coding changes were to be 
made. She tasked SEMA with a thorough analysis of the system, to ensure that, when 
changes and fixes were eventually made, the maintenance programmers would fully 
understand how the system worked and what effect their changes would make (Lacy, 
1992). In addition, a configuration control system was implemented to track Automated 
Data Service Requests (ADSR) and an error identification and tracking system established 


using Problem Tracking System Reports (PTSR). 


5. The Result 
This policy has paid great dividends. Ms. Lacy (30 June 1992) reports that 
errors have decreased and lines of code have been reduced from a high of over two 
million to about 1.36 million lines of executable COBOL code. More importantly for 
this analysis, thorough analysis of the current system has allowed SEMA personnel to 
do requirements analysis, database redesign, and product design for a proposed re- 


engineered RESFMS. 
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D. CONFIGURATION AND FUNCTION OF RESFMS 


Currently RESFMS contains four integrated subsystems: 


@ Active Duty Order Writing (AT/ADT) 
@ Inactive Duty Training Travel (IDTT) 
@® Travel 
@® RPN Accounting 
The AT/ADT, Travel, and RPN Accounting subsystems are fully implemented on 
a UNISYS 1100/92 mainframe computer operated by Navy Computer and 
Telecommunications Station (NCTS) New Orleans.’ Interface with the central 
mainframe is via microcomputers, emulating UNISYS terminals, located at 179 Reserve 
Sites throughout the United States. Typically, a Zenith 248 computer, equipped with an 
emulator board, functions as a dumb terminal connected via modem to leased telephone 
lines, operating at 9,600 bits per second (bps). These leased telephone lines feed directly 
to the UNISYS 1100/92 mainframe at NCTS, New Orleans. 
IDTT is processed by stand-alone modules on microcomputers at all of the 353 
Reserve sites throughout the United States. Processed data from the IDTT modules in 
the field is passed via modem and dial-up telephone lines to the mainframe-based 


subsystems of RESFMS for further processing or storage. 


*Navy Regional Data Automation Center, New Orleans (NARDAC) recently changed 
its name to Navy Computer and Telecommunications Station, New Orleans (NCTS). 
The organization and equipment referred to as part of the original development of 
RESFMS are the same, but the name has changed. 
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1. AT/ADT Subsystem 

The AT/ADT subsystem of RESFMS is currently used to issue approximately 
300,000 sets of active duty orders per year. Originally, the Active Duty Order Writing 
subsystem issued Annual Training (AT) orders for only Selected Reservists (SELRES), 
that is, those in an active drilling status in a reserve unit. Currently, RESFMS allows 
on-line request and subsequent printing of Annual Training (AT) and Active Duty 
Training (ADT) orders for SELRES, Individual Ready Reserve (IRR), and Health 
Science Educational Training Command (HSETC) personnel. Orders may be requested 
either individually or in batch. Program managers at Reserve Headquarters approve the 
requests then pass them electronically to Travel as appropriate. After travel 
arrangements have been made, orders may be printed at the requesting site. Under 
certain circumstances, orders may be printed within minutes of making the request. This 
feature is called 3-minute Orders. 

In addition, AT/ADT performs many management functions related to issuing 
and accounting for active duty orders. The system tracks and routes the request for 
orders through verification, approval, travel arrangements, accounting, and other 
appropriate stages of order generation. Modification and cancellation of order requests 
are also processed. Program managers are assisted through the tracking of days of active 
duty allotted (budgeted) versus those obligated. A history of active duty performed is 


maintained for individual reservists and Retirement Points are calculated and transmitted 


ps 


to Inactive Manpower and Personnel Management Information System (IMAPMIS)*. 
Finally, RPN accounting transactions are generated for action by the RPN Accounting 


subsystem. 


2. IDTT Subsystem 

IDTT processes and generates approximately 200,000 orders and 
transportation requests per year for SELRES receiving training away from their normal 
drill site while in a drill status (not on active duty). Accounting is also provided for 
IDTT funds which are part of the RPN appropriation. A budget operating target 
(OPTAR) is issued to field activities for IDTT expenditures. The IDTT module, 
operating as a stand-alone system on a microcomputer, allows the local Reserve 
Commanding Officer to keep track of these funds while issuing IDTT orders. Status of 
funds is periodically passed to the central RESFMS mainframe via dial-up modem and 
batch reporting. The AT/ADT module processes IDTT order information and passes 
accounting data to the RPN Accounting module for financial update. If airline travel 
arrangements are required in conjunction with IDTT orders, the request for travel is 
transmitted to RESFMS Travel vial dial-up modem for processing. Travel arrangements 
are made and tickets disseminated in a manner similar to that used for travel with active 


duty orders. 


*IMAPMIS runs on mainframe computers at Defense Finance and Accounting Center 
(DFAS), Cleveland, Ohio. It contains the master records of all Reserve personnel and 
financial information. Among other functions, IMAPMIS is used to issue reserve drill 
pay checks mailed by DFAS Cleveland to individual reservists or transmitted to their 
bank accounts. Retirement and promotion information is also retained there. 
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3. Travel 

The Travel subsystem processes about 167,000 travel arrangements and 60 
million dollars of associated bills per year for travel associated with both active duty and 
IDTT orders. Commercial airline reservations and ticketing are arranged through 
Scheduled Airline Ticket Office (SATO). Tickets can be electronically transmitted to 
teleticketing machines at 44 major Reserve sites, or to airlines themselves for issue. A 
Government Transportation Request (GTR) or a Request for Transportation Services 
(RTS) may be generated as appropriate. The system assists the Reserve Headquarters 
staff in selecting transportation that both meets operational needs and is the lowest cost 
which can be obtained at the time of ticketing. Thus, transportation requests are 
processed for Government Transportation System (GTS), or Military Airlift Command 
(MAC) flights, for commercial airline flights, charter bus, commercial bus, and rail 
(ransport. 

An important benefit of the Travel subsystem is that it supports Ticketing 
Adjustment and Unused Ticket Recoupment. Thus, if a ticket is issued for official travel 
and not used, the Travel subsystem allows expeditious cancellation of the ticket, 
recoupment of funds, and reallocation of resources. Over ten million dollars in travel 


funds were recouped and reused in FY91 alone (Lacy, 30 June 1992). 


4. RPN Accounting 
Full financial accounting and fund execution management for the $700 million 
Reserve Personnel Navy appropriation are provided by the RPN Accounting module of 


RESFMS. In addition to accounting transaction entry and processing, it provides general 
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ledger posting for Commitments, Obligations, and Accounts Payable. RPN Accounting 
tracks the RPN appropriation execution versus the budget plan. It provides numerous 
reports for use within the Naval Reserve as well as those required by outside agencies 
such as Financial Information Processing Center (FIPC) New Orleans. An on-line ad 
hoc inquiry capability is available to COMNAVRESFOR Finance (Code 06), Manpower 


(Code 02), and to FIPC New Orleans. 


5. External Interfaces 
RESFMS also supports interfaces with six other systems: 
@ Inactive Manpower and Personnel Management Information System (IMAPMIS). 
@ Micro Claims Processing System (MCPS). 
@ Reserve Headquarters System (RHS). 
@ Integrated Disbursing and Accounting Financial Management System (IDAFMS). 
® Centralized Expenditures/Reimbursement Processing System (CERPS). 


@ Navy Standard Claimancy Accounting Module (NSCAM). 


Figure | on the following page shows the current configuration of RESFMS. 


E. REASONS TO REDESIGN 


1. Size and Inefficiency of Current Software Configuration 
Extensive software maintenance efforts have reduced the size of RESFMS to 
1,360,000 lines of code (LOC). However, the maintenance patching process combined 


with an initial poor design of some modules have caused the system to still be difficult 
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Figure 1 Current RESFMS Configuration. 


to maintain (Furrey, 7 June 1992). COMNAVRESFOR spent $1,777,000 in FY90 and 
$1,608,000 in FY91 on software maintenance for RESFMS. Systems maintenance 
programmers and analysts estimate that a well designed rewrite of RESFMS should only 
be between 450,000 and 720,000 LOC or 33% to 52% of the current size (Lazar, 16 
July 1992).° Program managers estimate that such a system, programmed in an easy to 
maintain language, such as Ada, using modern programming practice and design, such 
as structured programming with loosely coupled, functionally independent modules, 
would require annual maintenance costing less than half of the current software 


maintenance budget (Furrey, 7 June 1992). 


2. Operating Costs for Hardware 

RESFMS is still running on the Unisys 1100/92 provided by NCTS New 
Orleans. Navy Industrial Fund (NIF) charges to COMNAVRESFOR for services 
provided by NCTS were $3,040,000 for FY90 and $3,514,000 for FY91. In addition 
to believing that these charges are excessive, COMNAVRESFOR managers feel that the 
service provided by NCTS should be rated as very poor (CNRF RESFMS Briefing 
Paper, August 1991). Response by NCTS personnel to problems is often slow. The 
charges for service are based on a fixed price contract (Blaylock, 1990). Thus, even if 
COMNAVRESFOR reprogrammed RESFMS to consume fewer computer resources, 


charges for NCTS service would not necessarily decline proportionally. This leaves 


°The manner in which these estimates were derived will be explained in Chapter VI 
in the discussion of costs considered for RESFMS. 


ay 


COMNAVRESFOR few alternatives to reduce the operating costs of RESFMS, which 
they have identified as excessive, if they continue to use NCTS operated hardware 


support. 


3.  Telecommunication Costs and Inadequacies 

Telecommunication charges make up almost a quarter of the RESFMS total 
Operating expenses (CNRF RESFMS Budget Expenditure, 14 May 1992). 
COMNAVRESFOR paid $2,253,000 in FY90 and $1,991,000 in FY91 to AT&T and 
NCTS for communication charges related to RESFMS. These charges result from the 
fact that COMNAVRESFOR leases dedicated communication lines connecting 179 sites 
across the continental United States to the Unisys 1100/92 at NCTS New Orleans. 
RESFMS is currently programmed so that microcomputers at the remote sites function 
only as dumb terminals and all processing is done centrally by the mainframe. Every 
menu, prompt, screen, and report is generated by the Unisys 1100/92 in New Orleans 
and transmitted over the leased telephone lines to the remote sites. Data transmission, 
at 9,600 bps, is very slow by today’s standards® and the potential processing power of 
the remote microcomputers 1s ignored in the present configuration. Also, 175 of the 354 
Naval Reserve sites are not connected to RESFMS, because the cost of connecting and 


maintaining dedicated data lines to these sites is prohibitive. 


°NAVNET and DDN use 56 kilobits per second lines, while FTS 2000 offers T1 
lines running at 1.54 megabits per second. 


28 


4. Integration With Other Naval Reserve Systems 

Since the inception of RESFMS, the Naval Reserve has developed a number 
of new information systems. Sensing both the potential benefits and problems associated 
with multiple system operation, in 1986 the COMNAVRESFOR Director of Information 
Systems (Code 10) drafted, and the Commander, then RADM Smith, adopted the 
Reserve Command Management Information Strategy (RESCOMMIS), a comprehensive 
plan for the development, operation, and maintenance of information technology in the 
Naval Reserve. Part of this strategy involves integration of all Reserve systems as well 
as the pursuit of modern technologies and procedures. 

The first system developed completely under RESCOMMIS was the Reserve 
Standard Training Administration and Readiness Support system (RSTARS). 
Development and implementation of RSTARS has been successful (Rautenberg, 15 
September 1991). RSTARS is a microcomputer based distributed process connected via 
dial-up modem to a centrally managed master database. The master database is 
maintained by the Reserve Headquarters System (RHS) which runs on a cluster of DEC 
VAX minicomputers and uses a Sharebase database machine for data storage. This 
hardware is physically located in COMNAVRESFOR Information Systems (Code 10) 
facilities in east New Orleans. At each Reserve site throughout the U.S., a stand-alone 
module of RSTARS manages the personnel files, unit assignments, promotions, 
attendance records, pay records, training requirements, and other administrative data 


required for Selected Reservists. Modifications to this data are sent, in the form of 
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change transmittals, to RHS for validation and storage. RHS validates all changes, and 
provides the interface to other Navy MPT systems requiring the data, such as IMAPMIS. 

The interface between RHS and RESFMS is not a continuous connection. 
In spite of the fact that much of the personnel data stored in RSTARS and RHS is exactly 
the same as that required by RESFMS, there is no sharing of this data. Changes to 
personnel data which affect RESFMS operation are sent twice a month from RHS as an 
update to the RESFMS database. It is not unusual for a Reserve site to have newly 
assigned reservists who wish to perform their annual active duty for training (AT). 
Frequently, personnel data required by RESFMS for order generation has not yet been 
received from RHS and a headquarters staff representative must manually enter the data 
into RESFMS to allow the request for orders to proceed. In my last assignment I was 
the Manpower Department Head at a major reserve site. This lack of interface between 
RESFMS and RHS caused what I considered to be a significant burden on my 
administrative support personnel trying to get active duty orders for Selected Reservists. 
It is not just a problem in the view of users in the field, however. COMNAVRESFOR 
information systems personnel believe that the data redundancy between RESFMS and 
RHS creates problems with data integrity and consumes computer mass storage capacity 


that could be better used for other Naval Reserve applications (Furrey, 14 May 1992). 


F. DECISION TO REDESIGN 
COMNAVRESFOR managers decided in 1991 that a redesign of RESFMS should 


be seriously examined. As a result of their success with RSTARS, they believed a 
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similar configuration could be employed with RESFMS. The active duty order request 
process could be reprogrammed as a module of RSTARS, or at least be compatible with 
RSTARS. Using the power of microcomputers already located at reserve sites, request 
entry, error checking, and request formatting could all be done off-line. The formatted 
request could then be transmitted as a transaction via dial-up modem, or sent over a 
packet switching network such as the Defense Data Network (DDN). As a result, 
dedicated data lines would no longer be needed and the telecommunication lease expenses 
could be saved. 

Experience with RHS has also shown that a large, data-intensive process could be 
programmed to run on a minicomputer while maintaining acceptable system performance. 
They concluded that the power of a mainframe was no longer required for RESFMS and 
that a network of minicomputers might provide a more cost effective replacement. In 
early 1991 study of Local Area Network (LAN) technology and minicomputers had been 
initiated to seek a solution for other Naval Reserve requirements and to meet goals set 
in RESCOMMIS. As a result of that study, a decision was made to purchase six AT&T 
3B2/600G minicomputers off government contract and establish an ethernet LAN at 
Naval Reserve headquarters in New Orleans. Experience with these computers and this 
LAN indicated that the 3B2/600G performed just as well or better than the VAX cluster 
running RHS. (Albro, 7 April 1992) 

In April 1991 AT&T acquired NCR (Karpinski, 22 April 1991). The newly 
merged company began offering NCR’s computing technology as upgrades to AT&T 


computers (Zipper, 10 December 1990)(NCR letter, 22 June 1992). AT&T had already 
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established its 3B2/600G on government contract, and so added the 3B2/GR as an 
upgrade to that computer on the contract. The AT&T 3B2/GR runs a RISC based 
processor and the UNIX operating system. It provides significant performance 
enhancements over the 3B2/600G. COMNAVRESFOR Code 10 made plans to upgrade 
all COMNAVRESFOR 3B2/600G computers to the 3B2/GR configuration as soon as 


possible (Albro, 20 May 1992). 


G. AUTHOR’S INVOLVEMENT 

In September 1991, I was asked to assist the Naval Reserve by performing an 
economic analysis of alternatives for RESFMS to possibly be used in budget proposals 
requesting funds for redesign and system acquisition. The goal of the redesign was to 
reduce telecommunication costs, reduce system maintenance costs, improve functionality, 
and improve system integration with other Reserve information systems. Possible 


alternatives were: 


@® Maintain the status quo 


@ Continue processing on the Unisys 1100/92 but change the telecommunication 
interface 


@ Redesign and recode the entire system to run on a network of minicomputers and 
use microcomputers as front end processors in the field 

The target system for complete redesign was to be the AT&T 3B2/GR minicomputer in 

order to maintain compatibility and consolidated maintenance support with minicomputers 


already in use. 
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H. TELECOMMUNICATION COSTS NO LONGER A FACTOR 

In April 1992 I learned of a decision by CNRF management that changed the 
alternatives to be studied for economic analysis. Approval had been received for a 
proposed Reserve Data Communications Technology Upgrade. This proposal involves 
the establishment of ethernet LANs at 33 major reserve sites. The 33 sites are 
Strategically located and suitably equipped so that the other 320 small reserve sites can 
connect to the LANs by use of a dial-up modem and regular telephone line. These LANs 
will be connected to a long-haul communications carrier via an AT&T 3B2/GR 
minicomputer operating as a gateway. The 3B2 runs UNIX as its operating system and 
the TCP/IP network protocols are built into UNIX, thus negating the need to purchase 
additional network management software. The long-haul communications are to be 
provided by NAVNET, a packet switching network connecting U.S. Navy commands. 
The LANs will improve interconnectivity at each site and, via NAVNET, will assure 
communication with headquarters. NAVNET was chosen because it provides the 
hctionality which the 1991 COMNAVRESFOR LAN study determined was required 
to meet RESCOMMIS data needs, and it is centrally funded, that is, no usage charges 
will be made to CNRF for packets transmitted on NAVNET. Communications expenses 
will be incurred only for the links from the Reserve site to the nearest NAVNET 
gateway. 

Knowing of the LAN project approval and NAVNET communications capability 
soon to be available, Mr. Tom Albro of CNRF began investigating the potential for 


connecting users to RESFMS on the Unisys 1100/92 using NAVNET and TCP/IP 
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protocols (Albro, 7 April 1992). Experiments conducted in May 1992 confirmed that 
completely satisfactory connectivity and functionality could be achieved using TCP/IP 
protocols on a packet switching network with little or no change to current RESFMS 
software (Albro, 20 May 1992). Therefore, a large portion of the telecommunications 
costs for RESFMS can be eliminated with no software redesign and no more hardware 
purchase than that already approved for the LAN project. This option will be pursued 
regardless of any decision on RESFMS redesign. Since reduction in communication 
charges will be the same for both the status quo and complete redesign, any consideration 
of costs or benefits associated with data communication are now irrelevant to this 


analysis. 


I. MAINFRAME TO MINICOMPUTER DECISION 

One significant hardware configuration question remains. Given the size and 
complexity of RESFMS, can it be moved to minicomputers? If yes, how many 
minicomputers will be required to provide acceptable capacity and performance? 

In order to answer these questions a measure of performance and capacity had to 
be found that could be meaningfully applied to both computer configurations and to the 
process as it currently runs. This kind of comparison is difficult. 

It is almost impossible to make general statements about different configurations 
of hardware running under different operating systems. Only the grossest kinds of 
comparisons can be made. Happily, it is really only the grossest kinds of 


comparisons that are necessary to determine whether there are significant hardware 
cost differences between different machine system populations. (Lorin, 1988) 
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Although a thorough analysis would require benchmark testing of CPU 
performance, knowledge of memory operations, page size, paging algorithms, and other 
details of system I/O, an adequate comparison could be made with simpler measures 


(Suh, 1991). It was determined that a comparison of five measures would be sufficient: 


@ CPU performance measured in MIPS 

@ Amount of configured memory (RAM) measured in megabytes 
@ [/O throughput measured in bits per second 

@ Mass storage capacity/requirements measured in megabytes 


@ Number of simultaneous users supported/required 


Performance data for the Unisys 1100/92 and for RESFMS was obtained from 
NCTS Pensacola, which maintains all performance data for NCTS computers throughout 
the southeast United States. Sales and technical representatives for AT&T were 
contacted to obtain performance and configuration data for the 3B2. 

The Unisys 1100/92 has two processors, each rated at 7.5 MIPS. The processors 
may be coupled together to yield a combined processing capacity of 15 MIPS. The 1100 
at NCTS New Orleans has 16 Megabytes of RAM, 8 Megabytes (MB) of which 1s 
configured for RESFMS. Maximum I/O throughput is 5.0 Megabits per second (Mbps). 
Mass storage may be attached in increments of 1.61 Gigabytes (GB). Maximum capacity 
depends on number of disk drives installed. Operators claim the system can support up 
to 1024 simultaneous TCP/IP users. A comparison with the AT&T 3B2/GR 1s provided 


in Table I below. 
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The AT&T 3B2/GR is rated at 25 MIPS. It may be configured with either 32 MB 
or 64 MB of RAM. COMNAVRESFOR is purchasing computers configured with 64 
MB of RAM. I/O bus speed, and therefore the maximum I/O throughput, is 5.0 Mbps. 
Each 3B2/GR may be configured with up to 50 GB of mass storage. However, 
COMNAVRESFOR is purchasing 3B2/GRs with a maximum of 15 GB per computer. 
Each 3B2/GR can support up to 256 simultaneous TCP/IP connections and, with UNIX, 
multiple computers can be connected, increasing the number of simultaneous users 
supported in multiples of 256. RESFMS performance statistics for the period September 
1991 through December 1991 show that RESFMS never used more than 12% of the 
Unisys 1100/92 CPU capacity. Mass storage for RESFMS required between 11.0 GB 
and 13.8 GB. The number of simultaneous users, with the present software architecture 
and configuration, was as many as 340. 


Table I COMPARISON OF UNISYS 1100/92 AND AT&T 3B2/GR 


Unisys 11/92 AT&T 3B2/GR 


Processor Speed 15 MIPS 25 MIPS 
RAM 16 MB 64 MB 
Maximum I/O Throughput 5.0 Mbps 5.0 Mbps 
Hard Disk Size 1.61 GB 2 GB 
Maximum Mass Storage Unknown 50 GB 
Mass Storage Configured 15 GB 15 GB 
Simultaneous Users 1024 256 
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From the above statistics it appears that the 3B2/GR is more capable than the 
Unisys 1100/92 in processing power and speed, and it can be configured to support I/O 
Operations in sufficient quantity and speed to meet the needs of RESFMS. Based on the 
analysis of current maintenance programmers, it can be assumed that a redesigned 
RESFMS will be smaller and not require as much mass storage or I/O throughput as the 
current system. Since the 3B2/GR is adequate to the current configuration numbers, it 
may be assumed it will be more than adequate for a redesigned system. The only 
concern will be support of simultaneous users. Yet, a stated purpose of the redesign is 
to change the system architecture to take advantage of minicomputer front-end processing 
of data. This change will greatly reduce the number of simultaneous users and negate 
concerns about the 3B2/GR’s ability to manage required communication connections. 

Thus, it may be concluded that a redesigned RESFMS could be run on a single 
AT&T 3B2/GR if necessary and that two 3B2/GRs running in tandem would provide 


better performance and reliability than the single Unisys 1100/92 does at present. 


J. MOST SIGNIFICANT ECONOMIC FACTOR 

With the questions of alternative systems configurations and capabilities settled, the 
consideration of costs and benefits become paramount. The costs of the current system 
may be readily obtained from COMNAVRESFOR records. Most of the costs of the 
proposed redesigned system are also straightforward. However, the cost of redesigning 
and reprogramming the software, as we will see in the following chapters, is both 


difficult to estimate and is crucial to this analysis. 
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IV. STRUCTURE OF ECONOMIC ANALYSIS 


A. DEFINING THE PROCESS 
The process of economic analysis has been described by Haga and Lang (1991) as 
"a systematic, six step procedure for comparing alternative means to meet an objective." 


The steps they define are as follows: 


Define the Objective 


@® Formulate Assumptions 


Choose Possible Alternatives 


® Compare Alternatives 


Perform Sensitivity Analysis 


The step of choosing possible alternatives is further divided into three distinct 


activities: 


@® Determine Costs 
® Determine Benefits 


@ Interface Costs and Benefits for Each Alternative 


In order to determine costs and benefits, we must know how to define them. In 


order to interface, that is compare, costs and benefits we must apply appropriate 
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economic analysis methods. Although the methods of comparison are similar for both 
public and private organizations, the analysis of public projects is "considerably more 
sophisticated than that for private sector projects." (Lang, 1989) The reason public 
project analysis is more complex is because we deal not only with revenues and costs, 
as in private projects, but also with benefits. 

Since the economic analysis of RESFMS concerns a U.S. Government, and thus 
public, project, it is imperative to obtain a precise definition of costs and benefits. It is 
also essential to determine which economic analysis procedures are applicable and how 


to apply them. 


B. COSTS 

Cost in the public sector can be defined as "a cash expenditure for operating, 
maintaining, and administering a public project." (Lang, 1989) Cost may also be viewed 
as inputs or flows of resources into the project, whereas benefits are outputs or results 
of the project (Haga and Lang, 1991). 

It is important in any economic evaluation to include all potential costs of each 
alternative in the analysis. DoD Instruction 7041.3 directs that in evaluations of 
alternatives for projects within DoD "costs of each alternative will be exhaustive". It 


goes on to delineate three categories of costs to be included: 


@ Research and Development (R&D) 
@® Investment Costs 


@® Recurring or Operations Costs 
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These categories of costs apply to all types of development and acquisition projects 
within DoD. The project which is the subject of this analysis, the redesign of RESFMS, 
involves the conversion of an information system to new equipment and software. In the 
case of projects involving replacement, augmentation or conversion of existing 
information processing assets, the FIRMR directs that any cost that can be stated in 
dollars shall be included with four notable exceptions that shall not be included (FIRMR, 
1990):’ 

@ Conversion of existing software and databases that would be redesigned regardless 
of whether or not augmentation or replacement systems are acquired 

® Purging duplicate or obsolete software, databases and files 

@® Development of documentation for existing application software 


@ Improvements in management and operating procedures 


Since the proposed redesign of RESFMS will be a conversion project, these guidelines 
can be used to determine costs applicable to this analysis. 

In summary, costs for this analysis should be exhaustive, should include R&D, 
Investment, and Operations costs and should be expressed in terms of dollars. Software 
related costs that will be incurred regardless of the development decision are not to be 
included in this analysis, nor are costs for improved management and operating 


procedures. 


‘Examples of costs that should be included can be found in FIRMR Bulletin C-14. 
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C. BENEFITS 

In private sector project analysis, the negative value of costs can be compared to 
the positive value of revenues, or to the market-value of potential system products or 
results. Many public sector projects have few or no revenues to evaluate and the 
products or results of system operation are not traded in the marketplace, making a 
market-value approach useless (Quirin and Wiginton, 1981). Public sector project 
alternatives can be evaluated on their relative costs, but cost alone does not always 
provide an accurate analysis of system desirability. Therefore, a comparison of costs to 
benefits is often required. 

Lang (1989) defines the benefit of a public sector project as "a cash advantage or 
other favorable consequence flowing to the public." He goes on to say that benefits “are 
not difficult to identify, but are relatively difficult to quantify and to price." In this 
context, then, benefits could be described not only as outputs but also as "synonymous 
with results, effectiveness, utility, or performance." (Haga and Lang, 1991). 

Many benefits may be assigned cash values. For instance, the construction of a 
bridge may reduce the number of miles driven by commuters each day and thus save on 
the consumption of gasoline. Given the number of commuters, average miles of driving 
saved, average fuel consumption, and cost of gasoline, these savings to the public can 
be quantified in specific money terms. Reducing the commuter miles driven daily may 
also reduce the amount of air pollution in the city, making the air cleaner and the city 
a more pleasant place to live. How does one assigne dollar value to a "more pleasant 


place to live?" The answer may be that a monetary value cannot be assigned, that this 
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benefit is what Lang (1989) calls an irreducible benefit. If so, then a means of 
evaluation other than monetary valuation must be found. 

A number of techniques have been developed to compare and evaluate benefits in 
public sector projects. One method is to use Benefit Cost Ratio (Walker, 1991). Haga 
and Lang (1991) provide a methodology for both quantifying benefits and for dealing 
with non-quantifiable output measures. DODI 7041.3 describes a method of graphical 
comparison of benefits and costs. These methods are all effective ways of dealing with 
benefits that are irreducible to monetary terms. 

Yet, while in the past almost all benefits were handled as irreducibles, much more 
effort is being spent today on costing, or estimating the monetary value, of benefits 
(Lang, 1989). In fact, the Department of Defense under CIM has recently mandated 
that all benefits will be expressed "in cash terms so that realization of benefits can be 
monitored and audited" (DDI Memo, July 1991). If we express benefits in cash terms, 
we do not require any extraordinary method of economic analysis or comparison. More 
importantly from the management standpoint of CIM, if we have devised means to 
quantify benefits in dollar amounts, we can use those means to verify that the proposed 
benefits do, in fact, accrue from system implementation. 

Therefore, in the analysis of RESFMS, any outputs of the system which become 
part of the evaluation should be quantified in monetary terms in order to satisfy current 


DoD directives. 
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D. DISCOUNTING 

As stated earlier, the concept of the time value of money is at the heart of 
economic analysis. What Quirin and Wiginton (1981) call the “bird-in-the-hand" 
principle states simply that it is preferable to receive early benefits than later benefits. 
The process of discounting allows us to account for this preference. Discounting 
calculates the present value of a future cost or benefit. Present value is obtained by 
applying a discount factor to a cost or benefit. 

In the Department of Defense, a ten percent discount factor is to be used when 
evaluating investment projects (DODI 7041.3 and OMB Circular A-94). Although 
established in 1972, this rate is still considered to be representative. It is an estimate of 
the average, pre-tax, rate of return on private investment, after adjusting for inflation. 
Thus, the ten percent discount rate may be considered as "the weighted average 
Opportunity cost of taking money from the private sector." (Haga and Lang, 1991) 

When evaluating investment decisions in the Department of Defense we apply 
discounting by following a two step process. First, make all estimates of the costs, 
savings, and benefits in terms of constant, base year dollars. Second, compute the 
present value of all cash flows by applying a ten percent discount factor. (Haga and 


Lang, 1991) 


E. ECONOMIC LIFE 
The length of time over which a project will be evaluated, the economic life, is an 


important factor in any economic analysis. A period that 1s too short may unfairly 
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penalize alternatives that require high initial investments and may also hide the negative 
effects of alternatives that have high out-year costs. Choosing an economic life that is 
too long may incorrectly attribute value to alternatives that will be obsolete or worn out 
before the final years of the analysis. 


There are three factors that determine economic life (Haga and Lang, 1991): 


@ Mission Life - The period of anticipated asset need. 
@ Physical Life - The period the asset may be used before physically wearing out. 
@ Technological Life - The period the asset may be used before it becomes 
technologically obsolete. 
The economic life chosen for analysis of alternatives is usually the shortest of the 


mission, physical and technological lives. (Haga and Lang, 1991) 


F. ANALYSIS TOOLS 
Of the economic analysis tools available for public and private sector investment 
decision analysis, there are six techniques which are appropriate for evaluation of 


information systems. (Walker, 1991) 


1. Present Value Analysis (PV) 
This method determines each alternative’s costs as stated in terms of their 
present value. It requires all alternatives to be of equal economic lives. This procedure 
is to be used when the economic life of a project is more than three years (Haga and 


Lang, 1991 and DODI 7041.3). 
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Present value analysis is the primary economic analysis tool and its 
calculations become the basis for calculations by other analysis tools. When costs and 
benefits considered can be quantified in terms of dollars, present value analysis is the 
preferred technique. Other economic analysis tools complement present value analysis 
by providing means of comparing alternatives when they are of unequal lives, or when 
costs and benefits cannot be expressed in dollars. Additional information is also provided 
by some techniques amplifying the present value analysis results. However, unless a 
mistake is made, other methods will never contradict the results obtained by present 


value analysis. 


2. Uniform Annual Cost (UAC) 
When evaluating alternatives with unequal economic lives, the Uniform 
Annual Cost method may be used to rank the alternatives. Because it is based on present 
value analysis, if the alternatives evaluated have equal economic lives, UAC is redundant 


to present value analysis. (Walker, 1991) 


3.  Savings/Investment Ratio (SIR) 

This ratio computes the relationship between future cost savings and the 
investment required to obtain those savings. "Because saving is a necessary ingredient, 
you use this if, and only if, you have a status quo alternative." (Haga and Lang, 1991) 
DODI 7041.3 states that the Savings/Investment Ratio should be shown when evaluating 


cost-reduction investment proposals involving incremental costs. 
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4. Discounted Payback (PB) 

Discounted Payback simply measures the amount of time it takes for an 
alternative to pay for itself. It determines what period of time is required for the 
accumulated present value of cost savings to offset the total present value cost of an 
alternative. (Haga and Lang, 1991) Again, since savings are involved, this method may 


be used if, and only if, there is a status quo alternative. 


5. Break Even Analysis (BE) 

When an information system will have variable costs as well as fixed 
investment requirements, Break Even Analysis may be useful. Garrison (1988) describes 
Break Even Analysis as finding the point where a project’s total expenses equal its total 
revenue. This point will also be where the decision maker will be indifferent to whether 


the project should be undertaken or not. 


6.  Benefit/Cost Ratio (BCR) 
BCR computes the ratio between outputs (benefits) of a project and inputs 
(costs). BCR can be used to compare both quantitative and non-quantitative benefits. 
Of the six techniques judged applicable to information systems, this is the only technique 


that can be used to evaluate non-monetary benefits. 


G. SELECTION OF TOOLS FOR RESFMS 
One of the alternatives for RESFMS will be a status quo alternative and a chief aim 
of the redesign is cost savings. Therefore, both Saving/Investment Ratio and Discounted 


Payback would be appropriate analysis methods. Since costs and benefits will be 
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expressed in dollars, Present Value Analysis is appropriate and preferred as a primary 
evaluation tool. The chosen economic life will be equal for all alternatives, which would 
make Uniform Annual Cost redundant to Present Value Analysis. There are no costs that 
vary with work load. Thus Break Even Analysis is probably not appropriate. Recently 
published CIM policy (DDI Memo, 23 July 1991) directs that any benefits used in an 
economic analysis be valued in monetary terms. The primary purpose of the redesign 
is cost savings, not added benefit. The benefits that will accrue are difficult to quantify 
in monetary terms. A Benefit/Cost Ratio might be useful, but would be unacceptable to 
CIM reviewers unless the benefits were in monetary terms. Techniques for comparing 
costs and cost savings will probably be more than adequate for the analysis. Therefore, 
Benefit/Cost Ratio will not be used. 

The economic analysis of alternatives for RESFMS, then, will involve Present 
Value Analysis, Saving/Investment Ratio, and Discounted Payback analysis of each 


alternative using the standard DoD discount rate of ten percent. 


H. RISK ANALYSIS 
The economic analysis process is, by its nature, uncertain. No matter how 
conscientious one is in identifying and evaluating costs and benefits, the process must use 
estimates and estimates involve uncertainty. 
If the economic evaluation method used does not reflect this uncertainty, then every 
assumption built into an economic analysis is a “best guess" and the final economic 


result is a consolidation of these "best guesses." Making decisions on the basis of 
such "best guess" calculations alone can be hazardous. (Stermole, 1984) 
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Therefore, the last step of the economic analysis procedure is an evaluation of 
uncertainty, or risk analysis. 

A number of methods exist for risk analysis. They range from simple methods to 
highly complex simulations. The method generally preferred in private corporations in 
the 1980s was categorical ranking of risk (Strassmann, 1990). The types of risk are 
described by adjectives (such as high, low, moderate, disaster). Then the risk types are 
converted to numeric scales by assigning weights to each category. More elaborate 
evaluations can be made by increasing the number of risk indicators to be ranked. 

Sensitivity analysis is a means of evaluating the effects of uncertainty by varying 
various parameters and thus determining their effect on the economic evaluation results. 
(Stermole, 1984) First, computations are made with the best estimate of values for each 
variable. Then, by changing the values of variables within reasonable limits and 
recomputing the results, the effects of each variable on the final outcome can be readily 
seen. (Haga and Lang, 1991) Through sensitivity analysis, critical strategic variables can 
be identified for careful attention by the decision maker and, if the project is approved, 
for close observation by the program manager. 

Probably the most sophisticated method of analyzing risk, and one gaining in 
popularity, is financial simulation using computer models (Strassmann, 1990). 
Simulation requires that you be able to assign probability distributions to each major cost 
determinant (Haga and Lang, 1991). Several commercially available software packages 


exist that work on microcomputers as either stand-alone programs or as add-in features 
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to spreadsheet programs.* These software packages generally use randomly generated 
numbers and a Monte Carlo method (such as spinning an imaginary roulette wheel) to 
simulate the probability of occurrence of costs and benefits. When the simulation is 
finished, the relative frequency of the various values can be plotted on a chart or graph. 
This form of simulation and display of results can be useful in the analysis of project 
risk. 

Sensitivity analysis and risk analysis of economic evaluations in the Department of 
Defense has been encouraged for more than twenty years (DODI 7041.3). However, 
recently it has taken on new import. The Director of Defense Information has directed, 
as a part of CIM policy, that calculations of cost and benefit for information systems 
projects in DoD will be adjusted to reflect the financial impacts of risk (DDI Memo, 23 
July 1991). Therefore, in the analysis of alternatives for RESFMS, an evaluation of 


financial risk of the estimates will also be made. 


‘Some examples of simulation software packages for microcomputers are: Risk 
Analysis and Simulation (for DOS), and @Risk (add-in program for Lotus 1-2-3), both 
from Palisade Corporation, Newfield, N.Y., and Crystal Ball-Forecasting and Risk 
Management, Market Engineering Corporation, Denver, Colorado (for Macintosh). 
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V. PROBLEMS WITH SOFTWARE DEVELOPMENT COSTING 


A. COST ESTIMATION METHODS 

The cost of redesigning and reprogramming the software for RESFMS is the most 
difficult to estimate and the most critical to this analysis of all of the costs associated 
with the redesign alternative. The problem of devising accurate and reliable cost 
estimates for the development of software systems is not new, nor is it unique to this 
analysis. Four methods to estimate software development cost appear in the literature 
(Hihn and Habib-agahi, 1991): 

1. Price to Win 

The method dubbed by Boehm (1981) as "Price to Win" in the private sector 

involves making the cost estimate equal to that believed necessary to win the job. The 
Same reasoning may be applied in the public sector to those estimates made to equal the 
amount (or schedule) believed to be desired (or politically acceptable) by those who will 


approve the project. 


2. Analogy 
This method involves reasoning by analogy with known previous development 
efforts. The actual costs of completed projects are related to an estimate of the cost of 
a similar new project. Boehm (1981) points out that the major advantage of this method 


is that it is based on actual project experience. The disadvantage 1s that it is unclear to 
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what degree the previous project effort is actually representative of the effort required 


on the new project. 


3. Expert Judgement 
When expert judgement is used, one or more experts are consulted who use 
their experience and knowledge of the proposed project to estimate the effort, and thus 
cost required. The disadvantage of expert judgement is that it "is no better than the 
expertise and objectivity of the estimator, who may be biased, optimistic, pessimistic, 
or unfamiliar with key aspects of the project." (Boehm, 1981) The estimate obtained by 
expert judgement may also not be repeatable by any other estimator. 
4. Algorithmic Models 
These methods use one or more algorithms which produce an estimate of the 
costs as a function of some number of variables considered to be cost drivers. The 
algorithms may be manually applied or incorporated into an automated costing tool. 


Boehm (1981) describes the five most common forms of estimation algorithms: 


@ Linear models 


Multiplicative models 
@ Analytic models 
® Tabular models 


® Composite models 
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All algorithmic models require some measure of project size as an input. The 
advantage of algorithmic models is that they are objective, repeatable, and generally 
efficient. However, the estimate produced is no more accurate than the sizing inputs and 
cost driver ratings used, and this constitutes the main disadvantage of algorithmic models. 
First, it may be difficult to obtain an accurate size estimate at the time the cost estimate 
is most needed. Second, the cost driver ratings used must be validated and calibrated for 
the particular organization doing the development. None of these models is generic in 
the sense of being able to apply the model to all development sites universally without 


modification or calibration of some sort. 


B. COCOMO 

Probably the most well known and widely studied software cost model is the 
hierarchy of models called COCOMO, for COnstructive COst MOdel, described by 
Boehm (1981). Boehm’s hierarchy involves three models: Basic COCOMO, Intermediate 
COCOMO, and Advanced COCOMO. COCOMO is an example of an algorithmic 
model. Basic COCOMO is a Static, single-value model. It computes development effort 
(in man-months) and costs (in dollars) as a function of program size expressed in number 
of lines of code. Intermediate COCOMO expands on the basic model by using a set of 
cost drivers to modify the estimate. These cost drivers are numeric values derived from 
a subjective assessment of system, hardware, personnel, environmental, and project 


attributes. | Advanced, or Detailed COCOMO incorporates all the attributes of 
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Intermediate COCOMO and adds an assessment of the cost driver’s impact on each phase 
of the software development process. 

The phases of software life-cycle used in COCOMO are those of the waterfall 
model. Originally presented by Royce (1970), and widely used in the 1970s and 1980s, 
the waterfall model has been codified in U.S. Government and Department of Defense 
directives and instructions as the primary means of development and documentation of 
information systems. The seven project activity phases for which COCOMO computes 


effort are: 


@ Plans and Requirements 
@® Product Design 

@® Programming 

@® Integration and Test 

@ Development 


@ Maintenance 


COCOMO was developed as a result of an analysis of 63 software projects 
completed by TRW, Inc. (Boehm, 1981). Many algorithmic models are described only 
in general terms because all or part of the model includes proprietary information. 
COCOMO is explained in detail, with reference to both rationale and actual computation 
of each aspect of the model. COCOMO also provides effective tools for software project 


management throughout each phase of development. Thus, COCOMO has become the 
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centerpiece of software project estimation and management instruction, as well as the 


basis for other automated estimating tools. 


C. SOFTWARE SIZE METRICS 

Most of the algorithmic models for cost estimation use, as their basic input, a 
measure Of project size in terms of lines of code (LOC) or thousands of lines of code 
(KLOC). LOC has also been used extensively as a productivity measure to determine 
project progress and programmer efficiency. Those who favor using LOC as a measure 
claim that there is high correlation between LOC and software development costs, that 
LOC can be easily counted, and that "a large body of literature and data predicated on 
LOC already exists." (Pressman, 1992) 

Another term for LOC, used by Boehm (1981), is delivered source instructions 
(DSI), or thousands of DSI (KDSI). The slightly different terminology suggests one of 
the problems with LOC measures: although the measure appears to be objective, the 
definition of what constitutes one line of code, or one source instruction, is not always 
clear. This and other problems with the metric are discussed in detail by Jones (1986). 
The use of LOC as a metric also penalizes well-designed but shorter programs, and does 
not easily adapt to measuring nonprocedural languages such as Fourth Generation 
Languages (4GL). The most important problem for cost estimation, however, is that 
using LOC requires a level of detail which may be difficult to achieve at the time the 


estimate is required (Pressman, 1992). 
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D. FUNCTION POINTS 

An alternative to size metrics such as LOC is the measurement of software 
"functionality" or "utility." Function oriented metrics were first proposed by Albrecht 
(1979) while serving as Program Manager of the Application and Maintenance 
Measurement Program for IBM (Behrens, 1983). 

Albrecht’s basic premise was that all the functions present in an application can be 
measured by examining the factors which are the "outward manifestations of any 
application" (Albrecht, 1979). The procedure is to, first, list and count the number of 
external user inputs, outputs, queries, and the number of external and internal master 
files. 

Each of these categories of input and output are counted individually and then 
weighted by numbers reflecting the relative value of the function to the 
user/customer. The weighted sum of the inputs and outputs is called "function 
points." (Albrecht and Gaffney, 1983) 
The weighted sum is then adjusted for other development environment factors such as 
communication complexity and time criticality. A subjective judgement of these general 
environmental factors is converted into a numeric value called Degree of Influence (DI) 
which serves as the basis for a General Characteristics Adjustment (GCA) to the function 
point count. 

The final product of this process is a number that can be used as a relative measure 

of program functionality and complexity. By computing function points for various 


systems and comparing the amount of programmer and analyst effort required to produce 


the system, a metric can be found to estimate project cost. For instance, a given 
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Organization may find that average productivity is five function points per man-month 
effort. Ifa proposed project has 200 function points, this organization can expect to 
expend 40 man-months producing the application. By applying the locally determined 
cost per man-month, a dollar cost estimate can also be made. 

The advantage of function point analysis is that a far better picture of project 
function and complexity can be obtained than that achieved with LOC estimates. 
Function points also give a measure of programming effort independent of program 
language used. The following table (Pressman, 1992 and Jones, 1986), illustrates this 
fact by showing a rough estimate of the average lines of code required to build one 
function point in various programming languages: 


Table TW AVERAGE LINES OF CODE PER FUNCTION POINT 


Programming Language LOC/FP (Average) 
Assembly language 300 
COBOL 100 
FORTRAN 100 
Pascal 90 
Ada 70 
OBJECTIVE-C oo 
Fourth-generation languages (4GL) 20 
Code generators 15 


56 


Examination of the above data shows that one LOC of Ada provides approximately 1.4 
times the "functionality" as one LOC of COBOL (Pressman, 1992). In other words, a 
program written in COBOL taking (on average) 1,000 LOC should only take about 700 
LOC, or be only 70 percent as large, if rewritten in Ada. 

One disadvantage of using function points is that, although it may appear very 
objective and straightforward, counting of function points involves subjective judgements. 
Still, the advantages of function points as a software productivity, project estimation, and 


project management tool are considered by some to be significant (Behrens, 1983). 


E. RELIABILITY OF MODELS 

Boehm (1981) asserted that models such as COCOMO could be expected to provide 
an estimate of cost within 20% of the actual cost 70% of the time. In spite of great 
effort to improve cost estimation and develop new models, the statistics have not changed 
much today. Stephen Gross (1992), AIS Division Head of the Naval Center for Cost 
Analysis, related his work in validating a new automated cost model for DoD use called 
SASET for Software Architecture, Sizing, and Estimating Tool. He stated that the 
criteria for acceptance of this model was that its predicted cost should be within 20% of 
the actual system cost 80% of the time. To validate the model, researchers used 
historical data from 25 systems developed for the Department of Defense. Using perfect 
information of project size and complexity, as determined by project completion, the 


model could predict within 20% of the correct cost 80% of the time (Gross, 1992). 
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However, at the time a cost estimate is required, before the project begins, perfect 
information of project size and complexity is not available. One can easily see that if the 
size estimate used is 70% accurate and the model 80% accurate, we only have a 56% 
chance, or slightly better than even odds, of being within 20% of actual project cost. 

Therefore, the better our understanding of the design of the system, the more 
accurate will be our estimate of effort. Actually, this statement is true regardless of the 
method used for cost estimation. If we use an algorithmic model, a fairly detailed 
description of system design will be required to make a reliable estimate of lines of code 
required to implement the system. If function points are used, a detailed description of 
system requirements and an accurate estimation of all input and output is needed to make 
the calculation. Even expert opinion and analogy require some knowledge of system 
design for a adequate estimate to be made. The dilemma then becomes: how much 
project development does one complete before making an estimate of cost and effort 
required? Pressman (1992) states that no matter attractive it may be to delay estimation 
until very late in the project, this is not a practical option. He suggests instead that 
several techniques be used "in tandem, each used as a cross-check for the others.” 


(Pressman, 1992) This analysis of RESFMS will use just such an approach. 


F. THE SOFTWARE COST ESTIMATION PROBLEM IN DOD 
Current DoD policy requires that a business case or functional economic analysis 
be done prior to project approval. Part of the business case is a cost/benefit analysis for 


the proposed information system. Project approval must be received before funds are 
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budgeted for the project. In order to determine an estimate of project costs and benefits, 
system requirements and some level of product design must be performed. Yet, 
requirements analysis and product design are the first two phases of the project for which 
we are seeking approval. What then can be done? Or better, what is in fact done? 

The FIRMR, OMB circulars, GSA guides, and DoD directives all state that 
requirements analysis as a part of project planning should begin as early as possible. 
Agencies are encouraged to develop strategic plans for business activities and information 
resource (IR) needs. From strategic plans come tactical plans indicating specific IR 
systems required for mission fulfillment. When IR system development needs are 
identified, a program manager (PM) should be appointed and project planning should 
begin (GSA Overview Guide, 1990). DoD instructions reinforce this concept. DoD 
Instruction 7920.2 describes the AIS life-cycle management process, the milestones 
required and activities in each phase. Request for Milestone QO ends the Need 
Justification Phase and involves submission of a Mission Needs Statement (MNS). 

The MNS is approved at Milestone 0, and the DoD Component is authorized to 
initiate the Concepts Development Phase and to expend resources for the activities 
of that Phase as planned. (DODI 7920.2, 7 March 1990) 

The recognition of a need to expend resources in planning for future systems 
development is not confined to government publications and directives. Strassmann 
(1990) states that five percent of an organization’s information budget should be directed 
to development plans. Since Strassmann is now Director of Defense Information (DDD), 
one would think this opinion would be strongly reinforced in practice. Yet, in spite of 


the encouragement in GSA publications, the opinion of Strassmann, and wording of DoD 
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instructions, DoD budget management personnel will not allow any funds to be identified 
explicitly for IR projects until that project has been fully approved for development or 
redesign. 

This fact is evident in examination of the RESFMS budget. A strong argument can 
be made for the need to redesign RESFMS. Yet, because RESFMS is considered a fully 
deployed system, and no redesign has been officially approved, no monies are designated 
for any redesign activities, nor have they been for the last two years. If a budget line 
item for development had any value other than zero, Navy budget personnel would cut 
the organization’s total budget by that amount and zero out the line item. The reasoning 
would be that there was no authorization for expending such funds on a fully deployed 
system. 

This mentality 1s not unique to the Naval Reserve. Interviews with CAPT Faubel 
(1992), the comptroller for Commander in Chief Atlantic Fleet, and with David Spivey 
(1992) of the U.S. Army Corps of Engineers confirm that the reasoning is the same in 
their organizations as well. Gross, of the Navy Center for Cost Analysis, does not 
believe that this presents a problem. He believes that if a commander feels a project is 
needed the commander will find the money somewhere in his budget to perform the 
needed cost analysis. He also stated that, in his experience, major commands desiring 
to develop systems had "plenty of money” available for this purpose (Gross, 1992). 
Others do not agree. 

Both CAPT Faubel (1992) and Mr. Spivey (1992) acknowledged that it is a serious 


problem that money is not officially available until after a project is approved, yet a 
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portion of the first phases of project development must be done in order to obtain good 
cost estimates required for project approval. Neither, however, wished to see a DoD 
requirement to explicitly identify funds for this purpose in a budget, and especially not 
if a specific figure or percentage were tied to the requirement. Both preferred to spend 
funds out of operating expenses, under the heading of general project management, thus 
allowing more organizational discretion in the execution of the budget. William Curtis 
(1992) of the Software Engineering Institute (SEI) stated that this is indeed a problem. 
He felt that it is not just a problem in the Department of Defense, but also in industry. 
However, it is not recognized as a problem in industry. He felt that insufficient 
resources were being devoted in general to the activities required to perform good cost 
estimates for software development projects before decisions are made to proceed with 
those projects. 

Therefore, even though it is widely recognized that resources should be expended 
to provide the information required for a reasonable cost estimate before system 
development approval, these resources cannot now be explicitly identified in DoD 
budgets. The assumption of this discussion is that funds are available in the program 
management budget of every agency to support these activities. This assumption may 
not be true for all agencies needing to develop IR systems, especially not in times of a 
declining budget. In the case of the Naval Reserve, such excess funds do not exist. The 
Naval Reserve may not be unique in this regard. 

Unlike most DoD agencies, the U.S. Army Corps of Engineers receives both an 


appropriated budget and significant nonappropriated income from services provided to 
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other governments and outside agencies. The Corps of Engineers has been successful 
in information system development (Spivey, 1992). It has had a series of excellent 
commanders who understood the problems and advantages of information system 
development and who have been supportive of expenditures in support of re-engineering 
efforts in the Corps. In spite of this environment, and a long period of increasing 
defense budgets in the 1980s, Spivey (1992) stated that getting money to do proper 
requirements analysis and system cost estimates was still a problem--one that they have 
so far overcome, but still a problem. If an organization such as the Corps of Engineers 
experiences a problem with this, what then do agencies without the same resources and 


expertise do? 


G. RESPONSE TO THE PROBLEM IN DOD 
As a result of my experience and interviews with several information system 


managers who did not wish to be quoted, I believe agencies do one of several things: 


1. Take the Money "Out of Hide" 

Whether the money is really available or not, some agencies "bite the bullet" 
and expend resources originally designated for other activities to perform the work 
needed to justify and cost a proposed system or redesign. If the system desired is a new 
development, Operation and Maintenance (O&M) funds may be used--this at the expense 
of personnel or training. The House Armed Services Committee reported that the 
Department of Defense planned to spend about nine billion dollars on ADP systems in 


FY90 and that three quarters of that amount was to come out of O&M (HASC, 1 July 
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1989). It is reasonable to assume that if the services planned to spend this much O&M 
to continue projects already approved, they would be willing to hide the cost of 
requirements analysis and cost estimates by spending O&M funds for those activities as 
well. 

If the system exists but needs redesign, money may be diverted from software 
maintenance efforts to plan for the redesign. In the case of RESFMS, the poor initial 
design caused maintenance programmers to be forced to do systems analysis and design 
activities just to adequately perform maintenance. The results of that design effort have 


been used by the author as a basis of cost estimates. 


2. Seek Outside Help 
Some organizations seek outside help, either by paying for consultants, or by 
getting "free" assistance from other agencies or sources, such as Naval Postgraduate 
School thesis students. If consultants are hired, this option becomes the same as the 


first--funds are taken "out of hide" to hire them. 


3. Guess 
In order to meet the requirement for an estimate of system development costs, 
some organizations use a form of Analogy or Expert Judgement to provide an estimate. 
The accuracy of the reasoning depends on many factors including: historical data 
available from previous projects, capability and experience of the estimator, and 


resources that have been expended in developing requirements analysis and design 
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proposals. The resulting estimate may be fairly accurate, or nothing more than a wild 


guess. For reasons discussed below, the wild guess is more likely. 


4. Price to Win 

Many organizations adopt a "Price to Win" strategy and use a cost which is 
believed to be politically acceptable, that is, a cost figure that is judged to be what will 
be approved. The practice of estimating costs based on external constraints can also be 
found outside DoD. A study of cost estimation methods in actual use at Jet Propulsion 
Laboratories found this practice prevalent there as well: 

The cost estimating process can be seriously impacted by conditions of severe 
budget or schedule constraints. The result is that the estimator’s job becomes less 
one of estimating costs and more one of analyzing system and functional tradeoffs. 
In many cases, if there is strong motivation to accept the work, the job may be 
accepted under the assumption that any inconsistencies between requirements, cost 
and schedule will be resolved while the task is under development. ... Thirty 
percent of the respondents reported being budget constrained while 20 percent 
reported being schedule constrained. (Hihn and Habib-agahi, 1991) 

If the politically acceptable approach is taken, requirements are then 
developed to match the cost, but the report 1s written as if the opposite had occurred. 
If the writers of the report understand the reasoning of reviewers well enough, any 
examination of the reasoning will find no problem with the estimate, for the requirements 
listed and the cost estimate will be completely consistent, since the former are 
constrained by the latter. Unfortunately, the requirements may not describe what is 


actually needed by the organization. So, if the project is approved, the requirements will 


undoubtedly change when actual development begins. 
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5. Combination 
Many organizations use a combination of approaches depending on what 
resources are available, what the political climate is at the time, and what expertise is 


available. 


H. ACCURACY OF DOD COST ESTIMATES 

How accurate will these estimates be? A look at recent Congressional hearings and 
GAO reports suggests the answer. Reports of the House Armed Services Committee 
(HASC, July 1989)(Endoso, July 1992), House Appropriations Committee (HAC, August 
1989), House Committee on Government Operations (HOC, November 1989), and 
numerous GAO reports have documented serious problems with DoD information 
systems management. Cost overruns in the hundreds of millions of dollars have been 
noted. Reasons cited, in addition to poor management and poor communication within 


DoD, have included (HOC, November 1989): 


@ Inaccurate cost and benefit estimates 
® Incomplete historical cost data 
® Estimators untrained in cost estimation techniques 


@ Excessive requirements growth (changes) 


Although the Congressional and GAO reports did not make the connection between 
requirements growth and poor planning or bad initial cost estimates, the author believes 


that they are strongly related. If the estimate is flawed to begin with, those developing 
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the system will have no compunction about changing the requirements later to suit their 
needs. Although this is not the only reason for requirements growth, it is a contributing 


factor. 


I. REASONS FOR POOR DOD COST AND BENEFIT ESTIMATES 

The reasons for poor cost and benefit estimates and the reasons for cost overruns 
and management failures in Department of Defense information system projects are 
interrelated. There is no one single cause, and the several causes overlap in their effects. 


The reasons for these problems include the following: 


1. Funds Not Identified Explicitly 

Budget managers in DoD are seeing fewer and fewer discretionary dollars due 
to declining total budget authority and the existence of large fixed-cost operating 
expenses. The more items designated by Congress, or higher authority, as "fenced" for 
a particular purpose, the less budget flexibility the organizations have to meet their 
changing obligations. In this environment it 1s natural that budget managers would not 
want funds for information system development feasibility studies to be explicitly 
designated in the budget. Yet, if we do not identify these funds accordingly we will 
never have any way to tell how much we are really spending on proposed system cost 
estimates. More important, it is likely that litthe or no money will be spent for this 
purpose. In times of declining budgets, the undesignated funds in the budget will be 
spent for other contingencies. The question becomes one of what we are willing to give 


up in order to provide an analysis of needs for the future. Planning and cost estimation 
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activities will receive scant funding, the research done will be scarce, and the marginal 
results will be presented as if a full and adequate analysis had been done. If the ruse is 
not discovered and the project 1s approved, it is highly probable that serious cost 


problems will result due to extensive changes in requirements during development. 


2. Political Pressure for a Particular Cost Figure 
The House Committee on Government Operations identified a potential cause 
of major system cost overruns as "the establishment of arbitrary cost caps by senior 
management" (HOC, November 1989). These caps are usually unrelated to information 
technology (IT) system requirements. The problem still exists. In fact, the problem is 


even worse now in a period of dramatic decline in Department of Defense funding. 


3. Poor Historical Data 

Good cost estimation requires good historical data. Congress identified the 
"lack of credible empirical data about the operation to be automated" (HOC, November 
1989) as a major contributor to erroneous cost estimates in DoD. Not only is there a 
lack of data on the particular function to be automated, but there is little data available 
on the performance of DoD organizations producing similar projects in the past. 

One goal of the Software Engineering Institute (SEI) is to help organizations 
improve the software development process. In performing this function, SEI conducts 
Software Process Assessments (SPA) of organizations to determine their present level of 
performance and suggestions for improvement. A significant element required for 


organizations to progress to higher levels of software process performance is collection 
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of detailed historical and operational data on the software development process as it is 
performed in that organization. Asa result of SPAs conducted, members of SEI estimate 
that most DoD organizations are functioning at the lowest level of Software Process 
performance due, in large part, to lack of software process data collection and analysis. 


(Curtis, May 1992) 


4. Lack of Trained Estimators 

The lack of individuals trained in "industrial cost estimation techniques as 
applied to automated systems" (HOC, November 1989), identified by Congress in 1989, 
is still a problem. In response to this problem, DoD established the Information 
Resources Management College (IRMC) as part of the National Defense University. The 
IRMC has developed a syllabus and a number of tools and resources for project 
managers. Yet, the courses at IRMC are not required of information system project 
managers, nor is equivalent training required. Worse, the course has not been designated 
as officially satisfying training requirements for Materiel Professionals who will be 
managing software development projects. Commands wishing to send personnel to 
IRMC must pay for this training. 

SEI also offers outstanding training in software project management, software 
requirements engineering, and numerous other related skills. DoD personnel may receive 
this training at considerably lower cost than individuals from private industry, yet the 
cost is still significant. 

The training suggested by Congress and provided by both IRMC and SEI is 


not mandatory. No requirement exists for DoD organizations to have members who have 
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received this or equivalent training. More important, budget funding for training is often 
not available. Travel/training is one of the few discretionary portions remaining in 
current budgets. In times of budget decline, the travel/training portion of the budget is 
one of the first items to be cut. The result is that, even if organizations want to get 
training for their members, the funds are often not available to pay for training fees or 
for the travel involved. 

An added problem with the lack of trained personnel is that, even when 
Organizations seek outside consultants they cannot be assured of a good cost estimate. 
One option for obtaining feasibility studies and cost estimates is to contract with another 
government organization’, or some commercial contractor to provide the study and 
report. If the organization contracting for the study has no members trained in cost 
estimation to oversee the process, there is no way to Judge the quality or applicability of 


siteereport received. 


5. Inadequate Cost Models and Tools 
In order to produce an adequate estimate, all the tools and methods available 
now require a reasonable estimate of project size and complexity as an input to the 
model. Also required is that the model be calibrated to a particular organization. Since 


we already know that, in DoD, few historical data are collected, few trained personnel 


*An example of a government organization that can provide needed expertise is the 
Office of Technical Assistance (OTA) of the U.S. General Services Administration 
(GSA). Within OTA are the Federal Systems Integration and Management Center 
(FEDSIM) and the Office of Software Development and Information Technology 
(OSDIT). Both can provide valuable assistance in evaluating, developing, and managing 
information systems. (GSA Overview Guide, 1990) 
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exist, and few funds are available for requirements analysis prior to cost estimation, the 


models will produce marginal estimates in their present form. 


J. THE DOUBLE DILEMMA 

Performing product design prior to cost estimation has disadvantages as well as 
advantages. In a rapidly changing environment, postponing design allows planners to 
consider many options. In recent years rapid technological advances have been the 
norm. Improvements have been made in hardware performance and in software 
languages and tools available. Postponing detailed design keeps the project flexible, but 
produces unreliable cost estimates. Performing detailed design prior to cost estimation 
provides much better cost estimates but “locks in" a system architecture that may be 
inadequate or obsolete by the time the system is developed. This double dilemma 1s 
faced by all information system developers, but it is most acute in DoD because of the 
long, mandated approval and development process. 

Most large projects for the Department of Defense have taken several years to 
develop. For instance the House Government Operations Committee (1989) reported on 
four major DoD systems that had been in development for over eight years. The process 
of approving the project prior to development is also complex and time consuming. 
Thus, if a detailed design is required as an input to a cost estimate, if a cost estimate 1s 
required prior to project approval, and if the approval process takes a matter of months 
or years, the whole design could be obsolete by the time development begins. Also, 


since the DoD AIS management process is based on the waterfall model, it 1s difficult 
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to take advantage of newer development models such as Incremental Development, 
Evolutionary Prototyping, and Rapid Prototyping. If anew development model is used, 
documentation to support the old waterfall method must still be generated. This 
documentation requirement increases the effort required and extends the time necessary 


for development. The process needs to be changed. 


K. RECOMMENDATIONS TO IMPROVE SOFTWARE COST ESTIMATION 
IN DOD 
Measures which, if taken, would mitigate or eliminate the problems identified are 


as follows: 


1.  Legitimize Expenditures for Feasibility Studies 

In order to ensure that proper planning and estimation is being done, the 
expenditure of funds should be explicitly authorized for feasibility studies prior to project 
approval. The approval of Milestone 0 and the Mission Needs Statement should 
automatically trigger a non-zero line item entry in the organization’s budget for research 
and development of the system. For fully deployed systems needing redesign, a similar, 
but different milestone system should be devised that would authorize funds to be 
explicitly identified for redesign and development of the follow-on system upon the 
approval of Redesign Milestone 0. Specific figures should not be mandated, but 
organizations should be encouraged to allocate five percent of their information systems 


budget to planning for future system development. This authorization should be included 
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in pertinent budget directives as well as in Life-Cycle Management directives, so that 


budget managers will recognize the legitimacy of this funding. 


2. Streamline the Development Process 
DoD directives should be changed to allow for new development models such 
as Incremental Development and Evolutionary Prototyping. The documentation 
requirements should be modified to reflect the differences in development models. 
Automated tools should be developed or acquired to assist in and speed up the design, 


evaluation, and development of systems. 


3. Mandate Training and Provide Funding to Support It 

Sufficient numbers of personnel trained in information system management 
and cost estimation techniques will never be available until both the training is required 
and funding is provided to obtain it. Congress has been inclined to cut DoD budget 
authorizations for information systems when problems with program management and 
cost overruns have occurred. Citing unsatisfactory progress, Congress recently targeted 
$300 million of ADP funding for possible cuts (Endoso, 1992). Congress itself 
identified lack of training as a problem in 1989 (HOC, November 1989). Therefore, 
DoD should ask, and Congress should approve the restoration of targeted ADP funding 
with a stipulation that the money is to be used for travel and training expenses for 
software cost estimation and software management. DoD should change policies and 
directives to indicate that IRMC training and equivalent courses (such as SEI and Naval 


Postgraduate School) are allowed and required of all software project managers and their 
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assistants. The goal should be to ensure that trained personnel will be present, by the 


end of FY93, in every DoD organization that performs development. 


4. Mandate Historical Data Collection and Set Standards 

Collection of historical data on software development efforts 1s essential and 
should be mandatory for not only DoD agencies but also all contractors that work for 
DoD agencies. The data collected should be available for reference at the local 
command where it was gathered and as part of a central DoD repository. Contractor 
data should be supplied to the database as part of contract deliverables. In order for this 
to be effective and not be onerous, DoD should not only stipulate required data format 
but also provide automated tools to aid in collecting the data. Ideally, historical data 
collection would be built into and integrated with project management tools. This would 
both increase the chances that the data would be collected and eventually improve the 


project management process. 


5. Improve the Cost Models 
Although implementing the previous four recommendations would 
dramatically improve the accuracy of current cost models, research should continue into 
improved costing models and tools. Several DoD organizations are attempting currently 
to provide better costing tools. SASET contains an example of an attempt to perform 
cost estimation in a new way. Developed by Martin Marietta Denver Astronautics 
division under contract to the Air Force Cost Center, SASET is a proprietary algorithmic 


model for cost estimation that operates in two modes. If size of the project in LOC is 
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available a cost estimate can be produced using conventional means. SASET is 
innovative, however, in that it has a mode in which a description of the function to be 
performed can be used to generate an estimate of effort required. The input required is 
not a function point analysis as described by Albrecht, but a description of module 
function such as: "Accounts Receivable Accounting Module." Based on a database of 
prior DoD projects, an average size and effort required to program a module performing 
that function is estimated. This is definitely a move in the right direction, but it needs 
to be refined. The database of DoD projects is too small and contains mostly real-time 
system development statistics. The model may be calibrated to a particular site, but that 
process presents the same problems as other cost models. 

If innovative cost estimation models can be developed to be integrated with 
the project management tools suggested above that also collect historical data, significant 
progress can be made toward achieving both accurate cost estimates and proficient 


program management. 
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VI. COST AND LIFETIME ESTIMATES 


A. COSTS CONSIDERED FOR RESFMS 

Two alternatives for RESFMS will be compared, a status quo alternative and a 
redesign alternative. Costs for the status quo alternative were determined by examining 
budget documents for RESFMS and through interviews with Mr. Ron Chamberlain, 
COMNAVRESFOR Budget Director (13 May 1992) and LCDR Furrey, RESFMS 
Redesign Project Manager (14 May 1992). To determine costs for the redesign 
alternative, cost estimation techniques had to be applied to data provided by 


COMNAVRESFOR staff and SEMA contractor personnel. 


l. Costs for Status Quo Alternative 

The largest single cost item for RESFMS in its current configuration 1s 
payment for services provided by NCTS New Orleans. NCTS provides both hardware 
operation and service to the Naval Reserve. In addition to operating the Unisys 1100/92 
mainframe computer, associated Input/Output devices, and mass storage, NCTS also 
provides four individuals to operate a help desk. The help desk accepts calls from users 
seeking information and assistance. The help desk personnel answer questions and refer 
problems to technical staff for corrective action. 

NCTS New Orleans is a Navy Industrial Fund activity. Such activities 
receive their initial funding from a revolving fund called the Defense Business Operations 


Fund (DBOF), formerly referred to as the Navy Industrial Fund (NIF). Because the 
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term DBOF is relatively new, most Navy personnel still refer to such activities as NIF 
activities. NIF activities perform services for other DoN or DoD activities and charge 
those customer activities according to the cost of providing the service. The customer 
activities pay for NIF services out of their appropriated funds budget. (Practical 
Comptrollership, January 1992) These monies are commonly referred to as "NIF 
charges." 

NIF charges for RESFMS operation were $3,040,000 in FY90 and 
$3,514,000 in FY91 (CNRF Ad Hoc Budget Expenditure, 1992). With optimistic 
estimates, the lowest projected figure for NIF charges in the next 6 years is $2,841,000 
in FY93 (CNRF Ad Hoc Budget Projection, 1992). From discussions with LCDR 
Furrey (14 May 1992) and Mr. Chamberlain (13 May 1992), I believe that prudent 
management may hold NIF charges between the FY90 and FY91 figures. Although the 
FY93 figure is possible, it is not probable that costs can be reduced to that degree. 
Inadequate data exist to determine a distribution of values other than these three points. 
Rough modeling can still be done, however, using a triangular distribution (Mendelsohn 
et al., 1991). Therefore, using a triangular distribution, I have selected $3,514,000 as 
the high value, $2,841,000 as the low value and $3,040,000 as the most probable, 
producing an expected value of $3,131,667 for NIF costs. 

The second largest cost factor in the current configuration is payment to 
contractors for software maintenance. In FY91, COMNAVRESFOR paid $1,608,000 
to SEMA, the contractor currently providing software maintenance for RESFMS. In 


FY92, because of DoD budget cuts, the software maintenance budget has been capped 
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at $1,050,000. Future DoD budget cuts may force further reductions in the RESFMS 
maintenance budget. The absolute minimum acceptable maintenance level for RESFMS 
in its current configuration is nine full time maintenance programmers (Furrey, 1992). 
COMNAVRESFOR pays SEMA $4,853 per man-month for services rendered. This 
figure includes all overhead and fringe benefits (Chamberlain, 1992). Nine programmers 
for one year at this rate would cost $524,124 and this is, therefore, the minimum 
acceptable maintenance budget. Using a triangular distribution with $1,608,000 as the 
high, $524,124 as the low and the current $1,050,000 as the most likely figure, produced 
an expected value for software maintenance of $1,116,393. 

On COMNAVRESFOR staff are five government civilian employees, two 
managers and three support personnel, who are assigned to RESFMS. Total cost for 
civilian personnel, including fringe benefits, is $307,000 (CNRF Ad Hoc Budget 
Execution, 1992). Expenses for supplies, including software provided to contractors, 
diskettes distributed to field sites, and office supplies was $100,000 in FY91 (CNRF Ad 
Hoc Budget Execution, 1992). Cost to the Naval Reserve of operating computer and 
support equipment at COMNAVRESFOR headquarters in support of RESFMS was 
$60,000 (CNRF Ad Hoc Budget Execution, 1992). RESFMS management and support 
personnel are required to travel in order to provide training for users and to attend 
management meetings in Washington, DC. Travel associated with RESFMS cost 
$34,000 in FY91 (CNRF Ad Hod Budget Execution, 1992). I believe these figures are 
representative of what can be expected in future years. Expected annual costs for the 


Status Quo Alternative which will be used in this analysis are summarized in Table III. 
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Table II RESFMS STATUS QUO COSTS 
a a Se 


NIF Charges to NCTS $3,131,667 
Software Maintenance (SEMA) $1,116,393 
CNRF Civilian Personnel $307,000 
Supplies $100,000 
Operations at CNRF $60,000 
Travel $34,000 
Annual Costs $4,749,060 


2. Costs for Redesign Alternative 


a. Software Cost Considerations 

The largest single cost item for the redesign of RESFMS 1s the cost of 
software development. After s lementation, the largest cost in any single year 
will be software maintenance expense. In order to determine the amount of these costs, 
software cost estimation methods, such as those discussed in Chapter V, must be used. 
To determine the cost of software development, an estimate of effort in terms of man- 
months is calculated. The number of man-months required is then multiplied by the 
known cost of a contractor man-month to produce a dollar cost estimate. 

Also important to economic analysis is the length of time required for 
the development effort, or the number of schedule months, since this figure affects the 


amount of lead time required. Nine man-months of effort can be accomplished in nine 
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months by one programmer, or in one month by nine programmers. Distribution of 
effort in a software project has been shown to take on the shape of a classic curve first 
described analytically by Lord Rayleigh (Pressman, 1992). Norden (1980) analyzed 
empirical data from software development projects to confirm that the Rayleigh curve 
describes the optimum distribution of effort. Many software cost estimation tools 
incorporate Rayleigh curve calculations of distribution to determine an optimum schedule 


for development of the project being estimated. 


b. COMNAVRESFOR Estimation of Development Effort 

When the redesign of RESFMS was first proposed, in mid 1991, 
COMNAVRESFOR management personnel asked SEMA contractors for an estimate of 
effort and schedule required to reprogram the system. The requested estimate was 
constrained by the assumption that only nine programmers would be used for 
development and that this number would remain constant throughout development. The 
target configuration for RESFMS was also not fully determined at the time of this 
estimate. Based on expert opinion of maintenance programmers, analogy with similar 
projects performed by other SEMA programmers, and the constraints given, an estimate 
of 77 schedule months for nine programmers, or a total of 693 man-months of effort was 
produced (CNRF RESFMS Briefing, August 1991). In June of 1992, after target system 
configuration had been decided, programming language and tools chosen, and initial 
system design nearly complete, the estimate was revised to 42 schedule months for nine 


programmers, or a total of 378 man-months of effort. 
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For purposes of this analysis, I chose to derive an estimate of 
development effort and cost by independent means from those used _ by 
COMNAVRESFOR staff personnel. Using sizing and functional data provided by 
COMNAVRESFOR and SEMA personnel, I have produced estimates with three different 


cost estimation tools. 


c. Function Point Estimate 

Two of the 18 software maintenance programmers and analysts for 
RESFMS have been tasked with preparing for RESFMS redesign. To date, system 
requirements, preliminary design, and product design are essentially complete (Furrey, 
14 May 1992). Using this data, SEMA personnel have calculated function points (FP) 
for the redesigned system using procedures devised by Albrecht (1984) and a program 
for IBM compatible personal computers written in Pascal by SEMA programmers. The 
result of their analysis for each of the three major modules of RESFMS is shown in 
Table [V (Lazar, 16 July 1992). 


Table [LV UNADJUSTED FUNCTION POINT COUNT 


RESFMS Module Unadjusted FP 
AT/ADT 763 
Travel 655 
RPN Accounting P22 
Total System 2670 
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If we assume that relative functionality should be an indicator of relative 
program size, examination of the above data indicates that AT/ADT should be 
approximately 61% and Travel 52% as large as RPN Accounting. 

To obtain the final function point value, the initial function point count 
(FC) is adjusted for general characteristics (GC). There are 14 general characteristics 
and each may have a degree of influence (DI) value between zero and five. DI is 
subjectively determined for each of the 14 general characteristics and these values are 
summed to produce a single GC value. The final function point measure (FP) is 


determined by the following formula (Albrecht, 1984): 


PP = Feu sos FT COOL GC) | 


SEMA analysts determined that GC for the redesigned RESFMS should 
be 39 (Lazar, 16 July 1992). Using this value in the formula above produces a function 
point value of 2776.8 for the redesigned system. 

Since determination of degree of influence for each of the general 
characteristics is subjective and descriptions given allow ranges of values (Albrecht, 
1984), different values for GC could be reasonably determined. I have gained knowledge 
of RESFMS system characteristics through my use of the current system in my previous 
two commands. Further, I have learned how the system will change in the redesign 
through interviews with Lacy (30 June 1992), Furrey (14 May 1992), and Lazar (16 July 
1992). Using this knowledge, I examined the descriptions of function point general 


characteristics (Albrecht, 1984) and calculated the possible GC minimum and maximum 
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values. Minimum GC was calculated by taking known characteristics of RESFMS and 
assigning the minimum suggested numeric value for degree of influence for each of those 
characteristics. Using this methodology a minimum GC of 21 was obtained. Repeating 
the process, but taking the maximum value suggested in each case, a maximum GC of 
55 was obtained. Using the minimum GC of 21 and unadjusted function point count 
provided by SEMA analysts, a function point value of 2296.2 is obtained for RESFMS. 
If the maximum GC of 55 is used, a value of 3204.0 is the result. 

If function points have been used at a software development activity 
consistently and historical data has been collected and retained for previous projects, a 
measure of effort required can be computed in terms of function points per man-month. 
Unfortunately, neither SEMA nor COMNAVRESFOR have collected and retained such 
consistent data. However, an estimate of effort to produce this system can still be made, 
but it will have to rely on rules of thumb for general industry averages and thus be less 
accurate than that derived from empirical data. To compensate for the lack of accuracy, 
I made several estimates in order to determine a feasible range for the desired value. 

Industry averages for productivity in terms of function points tend to 
fall between five and six function points per man-month of effort (Pressman, 1992; 
Zwieg, 9 May 1992). Use of fourth-generation languages or integrated Computer Aided 
Software Engineering (CASE) tools can increase productivity to 15 or 20 function points 
per man-month (FP/MM). Programmers recoding RESFMS will be using an integrated 
CASE environment called Ada Sage, which was produced under contract to the U.S. 


Department of Energy and is available at no cost to government agencies. The output 
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of Ada Sage is Ada programming code which is then compiled for the target hardware 
configuration. Even though integrated CASE tools increase productivity, manual coding 
is Sometimes required to adapt the output to a particular system configuration or function. 
Also, any new programming tool set or environment, no matter how easy to use, requires 
some amount of time for programmers and analysts to acquaint themselves with the 
features of the tool and become adept at its use (Zwieg, 9 May 19972). 

The RESFMS redesign project will be the first time that SEMA 
programmers have used Ada Sage and the first time they have programmed in the Ada 
language. The Navy Center for Cost Analysis has determined that organizations using 
Ada experience significant savings in software maintenance effort and expense as 
compared to those using other programming languages. Savings are also apparent in 
initial development efforts, for second and subsequent projects. However, a large 
penalty, in the form of training time, is imposed on organizations in their first use of Ada 
(Gross, 5 May 1992). Therefore, productivity gains associated with use of CASE tools 
will probably be negated by loss of productivity due to training time required for the first 
use of Ada. Considering all the above, I have used both the five FP/MM and six 
FP/MM productivity figures and the low, high, and SEMA estimates of GC to produce 
six possible values for RESFMS redesign effort using function points. These estimates 


are shown in Table V. 
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Table V TOTAL MM EFFORT BASED ON FUNCTION POINT ESTIMATES 





Source of Estimate FP Value 5 FP/MM 6 FP/MM 
Low GC Value 2296.2 459.24 382.70 
High GC Value 3204.0 640.80 534.00 
SEMA Estimate 2776.8 jo-00 462.80 





d. Estimate of Redesign Effort Using LOC 

As discussed previously, a number of models exist which use lines of 
code (LOC) as a size input to produce an estimate of effort required for software 
development. Two factors significantly affect the accuracy of a particular model’s 
estimate: whether or not the model has been calibrated for the particular organization 
performing the development and the accuracy of the LOC input estimate. 

Insufficient historical data exists to calibrate models specifically to 
Commander Naval Reserve Force or SEMA projects. However, the problem of 
calibration could be mitigated if models were used which had been calibrated to 
Department of the Navy or Department of Defense projects. Models so calibrated would 
be less accurate in this analysis than if they had been calibrated for the specific 
development organization, but considerably more accurate than models calibrated for 
civilian industry use. 

In the case of RESFMS, a Status quo system exists which may be used 


as a basis for LOC estimates of the redesign. In June 1992 a utility program was run 


84 


by SEMA maintenance personnel which counts lines of code in COBOL. This program 
was used to determine the exact LOC counts tor each RESFMS module. The results as 
reported by Ben Lazar (16 July 1992) were: 


Table VI RESFMS CURRENT CONFIGURATION LINES 
OF CODE 


Program Module LOC Total LOC 
RPN 275 ,668 
RPN PROC 60,831 
Total RPN 336,449 
AT/ADT Soyo 
Travel toh ee 
AT/ADT-Travel PROC 89,190 
Total AT/ADT-Travel 1,022,949 
Total RESFMS | Soe lhe 


The term PROC in the above table refers to modules of program code 
that are common support modules called, and used repeatedly, by applications programs. 
Thus RPN PROC is the LOC for support programs used by the RPN accounting 
application module, and AT/ADT-Travel PROC is a LOC count of support programs 
used in common by both the AT/ADT and the Travel modules. In order to compute the 
total LOC for each subsystem, the RPN PROC LOC should be added to RPN and the 
AT/ADT-Travel PROC LOC should be split evenly between AT/ADT and Travel with 


half added to each (Lazar, 16 July 1992). 
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Special note should be taken of the relative size of the modules. From 
function point analysis it was determined that the AT/ADT module should, ideally, be 
61% as large as RPN Accounting, and that Travel should be 52% as large as RPN. If 
it 1s assumed that the RPN PROC is counted as part of the RPN function and that the 
AT/ADT-Travel PROC may be equally divided among the two modules which share the 
PROCs, then RPN Accounting is 336,449 LOC, AT/ADT is 617,127 LOC, and Travel 
is 405,822 LOC. Thus, AT/ADT ts actually 183% and Travel 121% as large as RPN 
Accounting. Why ts there such a disparity between the actual figures and those one 
would predict based on function points? 

The answer lies in the way RESFMS was developed. The AT/ADT 
module and the Travel module were the first programmed. They were also both 
produced by NARDAC New Orleans. RPN Accounting was produced by civilian 
contractors. It 1s obvious from both historical data (Lacy, 30 June 1992) and current 
condition of the code (Lazar, 16 July 1992) that little or no design was done prior to the 
start of coding, that structured programming techniques were not used, and that huge 
amounts of code was redundantly copied throughout the modules. The AT/ADT and 
Travel modules are, therefore, considerably larger than optimum design would indicate. 
On the other hand, SEMA maintenance programmers assert that the RPN Accounting 
module is well designed and reasonably efficient (Lazar, 16 July 1992). RPN 
Accounting 1s probably 10% larger than optimum due to growth from software 
maintenance patches inserted during the last seven and a half years of operation (Lazar, 


16 July 1992), but is otherwise well programmed. Therefore, consensus of RESFMS 
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managers and programmers (Furrey, 14 May 1992)(Lacy, 30 June 1992)(Lazar, 16 July 
1992) is that the proportions calculated from function point analysis are very close to 
optimum for a redesigned system and the RPN Accounting module may be used as a 
baseline to determine what size the programs should be if written in COBOL. 

Using this reasoning I have developed both a Low and a High estimate 
for LOC to be used as input for cost estimation models. Both estimates assume that the 
relative size of the RPN PROC to RPN Accounting module is consistent with the amount 
of support programs to applications programs that will be present in the final system. 
The High estimate assumes that the RPN Accounting module is the correct size, as 1s, 
for the redesigned system and makes no adjustment to LOC estimate for RPN 
Accounting. In the High estimate, AT/ADT is computed at 61% of the size of RPN 
Accounting and Travel is computed as 52% of RPN Accounting. 

The Low estimate assumes that RPN Accounting is 10% too large at 
present and adjusts its size downward. Also, consideration is given for the fact that the 
target system will be programmed in Ada which is more efficient than COBOL. Recall 
that, on average, Ada requires only 70% as many lines of code as COBOL to produce 
the same relative functionality (Pressman, 1992). Therefore, I have further adjusted the 
RPN Accounting module downward to account for the use of Ada. If we adjust 90% for 
maintenance growth and 70% for Ada use, our final estimate will be 63% as large as the 
original (0.9 x 0.7 = 0.63). The result is that the estimate for RPN Accounting is 
reduced from its present size of 336,500 LOC in COBOL to approximately 212,000 LOC 


in Ada. The Low estimate takes this as the new baseline for RPN Accounting and 
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calculates AT/ADT as 61% and Travel as 52% of this reduced baseline. A summary of 
LOC estimates used for this analysis is found in Table VII below. 


Table VIL LOC VALUES USED FOR RESFMS COST ESTIMATES 
Ta ll I i a 


Module Low LOC Estimate High LOC Estimate 
RPN Wieaeahs) 276,000 

RPN Support 38,150 60,500 

RPN Total 212,000 336,500 
AT/ADT 106,200 168,250 

AT/ADT Support 23,300 37,000 

AT/ADT Total 129,500 205,250 
Travel 90,600 143,500 

Travel Support 19,900 31,500 

Travel Total 110,500 175,000 
RESFMS Total 452,000 716,750 


ee ny 

A third LOC estimate to be considered is a variation of the Low LOC 
estimate. One of the goals of the CIM initiative is increased reuse of software within 
DoD. The Director of Defense Information has stated that one goal of CIM is to achieve 
50% reuse of code in developing new systems. The RPN Accounting module of 
RESFMS is a large financial accounting program with accommodations to the unique 
aspects of Naval Reserve record keeping. It is very likely that up to 50% of the RPN 
Accounting module could be provided by reused generic accounting modules. This 


possibility would greatly reduce the effort required to redesign the entire RESFMS 
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system. To allow for the possibility of incorporating reused Ada programs in RESFMS, 
I have added a third LOC estimate which adjusts the Low LOC estimate to account for 
50% of the RPN Accounting module being constructed of reused code. 

The Naval Center for Cost Analysis, in Washington, DC, distributes 
two software cost estimation tools that use LOC as an input: REVIC, for REVised 
Intermediate Cocomo, and SASET, for Software Architecture, Sizing, and Estimating 
Tool. Both are available in versions which run on personal computers. Using these two 
cost estimating tools and the three LOC estimates described above, I have produced a 
series of estimates of effort required for development and for maintenance effort 


necessary to sustain a redesigned RESFMS. 


e. REVIC Estimate of RESFMS Redesign Effort 

REVIC is simply the Intermediate COCOMO model (Boehm, 1981) 
adapted for use in the Department of Defense. It is revised because it has been calibrated 
based on empirical data from DoD projects, and because it adds a new development 
mode, the Ada mode, to account for the difficulties of using Ada for the first time 
(Gross, 5 May 1992). The user 1s queried for values for 20 Environmental Factors, such 
as programmer capability and database size, that are used to compute the environmental 
modifier in the COCOMO model (Kile, 1991). Just as COCOMO allows for adjustments 
to its calculation for software modification efforts, REVIC queries the user to determine 
if a particular module is being produced as an initial development effort or as a 
modification of existing code. If modules are designated as being modified, REVIC asks 


what percent of redesign and recode will be required. The result of REVIC’s 
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calculations is an estimate of effort required, in man-months, and an estimate of the 
optimum number of schedule months. These estimates are displayed for each phase of 
development as well as a summary of the total effort. REVIC also computes an estimate 
of life cycle maintenance effort required for 15 years of operation. 

REVIC divides effort and schedule estimates into those required for six 


phases of system development: 


@ Software Requirements Engineering 
@ Preliminary Design 

@ Critical Design 

@® Code & Unit Testing 

@ Integration & Test 


@ Development Test & Evaluation 


Software analysts working on RESFMS have completed activities 
equivalent to the first two phases of the REVIC model, that is Software Requirements 
Engineering and Preliminary Design. Therefore, estimates used for this analysis and 
derived using REVIC have had the values for Software Requirements Engineering and 
Preliminary Design phases subtracted from the totals reported by REVIC. 

The redesign of RESFMS will largely involve recoding existing COBOL 
programs in a new language, Ada. Actual change in functionality will be small. SEMA 
analysts estimate that functional design will change only 10% to 30% from current 


functionality (Lazar, 16 July 1992). Even in the AT/ADT and Travel modules that are 
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so poorly coded, the actual design will change little. Almost all the gains in efficiency 
will result from improved coding practice and use of common modules instead of 
redundant code. Even though, in this analysis, the values computed for the Preliminary 
Design Phase will be discarded, it is important to accurately estimate the amount of 
design change in a COCOMO software modification cost estimate. This is because the 
design activity is not confined to those phases with "design" in their title (Boehm, 1981). 
Design activities occur in varying amounts in almost all phases. Thus, the percent 
change in design affects estimates of effort for all phases of development. For this 
evaluation I have chosen a design change value of 20% for use in all REVIC cost 
estimations. 

Results of REVIC calculations using the three described LOC estimates 
as input and subtracting values for Software Requirements Engineering and Preliminary 
Design phases were as follows: 


Table VIIT REVIC DERIVED ESTIMATES 


LOC Estimate Total MM Sched Mos 
50% Reuse in RPN 2a ae 25.5 
Low LOC Estimate 856.1 AAO) 
High LOC Estimate 504.9 DNS 


f. SASET Estimate of RESFMS Redesign Effort 
SASET was developed by Martin Marietta Denver Aerospace 


Corporation under contract to the Air Force Cost Center. The Naval Center for Cost 


oy 


Analysis has validated SASET for Navy projects using empirical data from 25 DoD 
software development projects (Gross, 5 May 1992). SASET uses proprietary algorithms 
for its calculations. An interesting feature 1s that LOC estimates can, and should, be 
divided up into systems programs, application programs, and support programs if 
possible. Also, whether the code is being initially programmed, modified, or ported to 
a new hardware platform makes a difference in estimates of effort required to develop. 
The more specific the user can be concerning the function, use, and development pattern 
of individual modules, the better will be the estimate that SASET produces (Silver et al., 
1990). In addition to a LOC estimate, the user is required to enter values for 32 factors 
concerning system characteristics, programmer proficiency, and development 
environment. The values for these 32 factors are used to determine both budget and 
schedule multipliers that affect the estimate of effort and schedule required. Output may 
be obtained in a number of forms. I chose to receive a measure of effort in man-months 
and optimum schedule in calendar months by phase of development. Maintenance effort 
required can be computed if the user so desires. 

The description of RESFMS PROC module functions (Lazar, 16 July 
1992) is close to, but does not exactly match, the description of support software in 
SASET estimation (Silver, 1990). To provide for the possibility of error in designation 
of software type, I have used SASET to calculate estimates both with support software 
(PROC modules) separate and with them lumped into the application LOC estimate. 
RESFMS redesign will not involve coding any systems software. In this case then, 


entries in SASET would be appropriate only for applications and support module LOC. 
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Since breaking out support software from applications software LOC involves two entries 
each for RPN, AT/ADT, and Travel modules, I refer to this entry format as "6 Module." 
Since lumping PROC modules with its associated application involves only one entry 
each for the three major modules, I refer to this option as "3 Module." 


Software development as defined by SASET is broken into ten phases: 


@ Systems Requirements 

@ Requirements Allocation 

@ Software Requirements 

@ Preliminary Design 

@ Detailed Design 

@ Code 

@ Checkout 

@ Unit Testing 

@ Physical and Formal Quality Testing, Integration 


@ Systems Test & Integration 


SEMA software analysts working on RESFMS redesign have already completed actions 
equivalent to the first four phases computed by SASET. Therefore, estimates used for 
this analysis and derived using SASET have had the values for Systems Requirements, 
Requirements Allocation, Software Requirements, and Preliminary Design subtracted 


from the totals reported by SASET. 
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In order to produce a reasonable range of estimates using SASET, five 
sets of input values were used: Low LOC with 50% reuse of code in the RPN module 
(called "50% Reused" in the tables and graphs), Low LOC 3 Module, Low LOC 6 
Module, High LOC 3 Module, and High LOC 6 Module. The results of these 
calculations are shown in Table IX. 


Table IX SASET DERIVED ESTIMATES 


LOC Estimate Total MM Sched Mos 
50% Reused 336.79 20 
Low LOC 6 Module 439.96 ZN 
Low LOC 3 Module 488.82 Ds, 
High LOC 6 Module 697.71 25 
High LOC 3 Module Selle Zo 


g. Summary of Software Development Effort Estimates 
Using the three cost estimation tools, a total of 14 estimates of effort 
were derived: three using Function Point Analysis, five using REVIC, and six using 
SASET. The estimates range from a low of 321.3 MM toa high of 775.15 MM. The 
mean value is 496.83, and standard deviation is 135.46. The mean value will be used 
in initial economic evaluation and the standard deviation 1s needed for risk analysis. A 


summary of the estimates of effort is found in Figure 2. 
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Figure 2 Estimates of Effort for RESFMS Redesign 
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h. Software Maintenance Estimates 

Both REVIC and SASET allow computation of the level of effort that 
will be required to maintain the system during its extended life-cycle. These values are 
generally expressed in terms of a full-time software person month (FSP). One FSP is 
the equivalent of one software maintenance person working one full month, and 3.5 FSP 
would be the equivalent of three and a half full-time software personnel working for one 
month on the project. The FSP figures can be multiplied by 12 to obtain an annual 
maintenance figure in man-months (MM) and this figure can be multiplied by the cost 
per MM to obtain annual maintenance cost. 

Since the 50% Reuse estimate results in a system exactly as large and 
complex as the Low LOC estimate, the maintenance required on such systems would be 
the same. Also, when using SASET, though dividing LOC estimates for modules into 
support and applications portions resulted in different estimates of development effort 
than lumping them together, it made no difference in the resulting maintenance effort 
estimation. Therefore, four sets of values were derived for FSP requirements for 
maintenance: two from REVIC and two from SASET. The mean and standard deviation 
of the four estimates was calculated for each year of maintenance. The mean of the 
maintenance FSP values was used in the initial economic analysis and the standard 
deviation used later in risk analysis computations. Values derived for maintenance are 


displayed in Table X. 
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Table X FSP ESTIMATES FOR SOFTWARE MAINTENANCE 


YEAR REVIC REVIC SASET SASET Mean Std Dev 
Low High Low High 


| On 9.8 2730 3.74 5.700 e202 
2 6.0 8.5 1.89 Zoe 4.845 2 o255 
3 oe ES 1.53 2.43 4.190 2 SONG 
: 4.6 6.6 1.30 2.06 3.640 2.42592 
5 4.6 6.6 1.06 1.68 3.485 2.58747 
6 4.6 6.6 1.00 ey 3.447 PEE IES: 
7 4.6 6.6 0.88 1.40 3.370 2.70966 
8 4.6 6.6 0.77 eg BS 2.78980 


l. Hardware Investment and Technical Support 

The target computer configuration for the RESFMS redesign is an 
AT&T 3B2/GR. Earlier analysis determined that RESFMS could run on a single 
3B2/GR if it were configured with at least 14 GB of mass storage, but that the number 
of possible simultaneous uses and limitations on network connections per computer would 
indicate that two 3B2/GRs would be optimum. Also, the operation of two computers for 
the same application would allow for system redundancy and backup in case the primary 
computer fails. 

Vendor representatives for the AT&T contract provided cost figures for 
the computers and vendor support that would be required (NCR letter, 22 June 1992). 
The configuration recommended includes the following items for each computer 


purchased: 


Jy 


@ 3B2/GR computer with RISC processor running at 25 MIPS 

@ 64 Megabytes of RAM 

@ 15 Gigabytes of hard disk storage 

@ Multi-Network Processor board to allow connection of 256 simultaneous users 
® Cables and connectors 

@ Storage cabinets for all equipment 

@ POSIX operating system software (a form of UNIX) 


@ TCP/IP network software for UNIX systems 


Total cost for this configuration is $129,892.47. Two computers would 
thus cost $259,784.94. These computers would require vendor supplied maintenance. 
Vendor maintenance for the hardware would cost $3,344.09 per year. Maintenance 
support for the software (operating system and network software) would cost $562.56 per 
year. AT&T will supply this maintenance support for up to five years. After that time 
support may be purchased from other vendors at a cost near, or below, that quoted by 
AT&T (Albro, 13 July 1992). 

COMNAVRESFOR is already operating a network of 3B2/GR 
computers used for other applications. Sufficient space, air conditioning capacity, and 
electrical power are available to easily add four to six new computers to this network. 
Addition of two new computers for RESFMS would not require any additional 
construction, nor would they use any significant amounts of additional utilities than those 


already in operation. Computer room operations support would require the addition of 
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the equivalent of one and a half technical support personnel to meet the needs of the 
additional workload. Cost of technical support personnel would be $68,000 per person, 
including overhead and fringe benefits (Albro, 13 July 1992). Cost of technical support 


for the new computers would thus be $102,000 per year. 


of Other Operational Costs of Redesign Alternative 

NCTS New Orleans currently provides the service of a help desk for 
RESFMS support. If the system were redesigned, this help desk function would have 
to be provided by COMNAVRESFOR. The help desk function would be provided by 
SEMA contract personnel and would, therefore, cost the Naval Reserve $4,852.40 per 
month per person. Four individuals would be required for the help desk function. 
Sufficient telephone and office space already exist at COMNAVRESFOR facilities to 
accommodate this function. Therefore, total additional cost would be $232,915.20 per 
year. 

Management and support personnel attached to COMNAVRESFOR and 
now assigned to RESFMS would continue this function with the redesigned system. No 
additional tasking would be required, but neither would their workload be reduced. 
Supplies and travel requirements of the current system should also remain fairly constant 
if a redesigned system begins operation. Since SEMA contractors are physically located 
in COMNAVRESFOR spaces, and system requirements and design are already 
determined, no additional travel should be required by development personnel. 


Therefore, costs for COMNAVRESFOR civilian personnel, supplies, and travel are the 
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Same as those found in the current system and would begin to be attributed to the 
redesigned system upon initiation of system operation. 

Software tools that will be used by programmers and analysts working 
on the redesign have either already been purchased or are available from other 
government agencies at no cost to the Naval Reserve. COMNAVRESFOR is providing 
all software tools for SEMA contract programmers. Much of the initial programming 
will be done on personal computers and then the code will be recompiled for the 
3B2/GR. This will allow much greater programmer productivity, since the 3B2/GR will 
only have to be shared for testing of finished program modules. COMNAVRESFOR 
already owns sufficient numbers of personal computers to support a full development 
effort on RESFMS redesign and will provide these computers to SEMA for their use. 
Therefore, no additional costs will be incurred for either software tools or development 


computer assets. 


B. BENEFITS CONSIDERED FOR RESFMS 

Even though benefits for RESFMS will not be formally or financially analyzed, it 
is important to verify that the benefits of the redesigned system are at least equivalent 
with those of the existing system. Otherwise, any cost savings that may accrue from a 
redesign may be mitigated by reduced functionality or reduced benefits. 

A briefing document prepared for COMNAVRESFOR (CNRF RESFMS Briefing 
Paper, August 1991) has identified the following benefits derived from the current 


RESFMS configuration: 
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@ Significantly Improved Management of RPN Funds 


® Fund Utilization Improved from 98% of 330M in FY82 to 99.97% of 683 M in 
FY87 


@ Allowed Reuse of $5M in Travel Funds Recouped from Tickets in FY88/89 
@ Improved Tracking of Active Duty Orders 
@® Reduced Time to Generate Orders 


@ Fully Certified Accounting System January 1987 


Since the redesigned system will have the same functional requirements as the 
current system, and will be programmed by personnel currently maintaining RESFMS, 
there is ample evidence to assume that benefits of the current system will be present in 
the redesigned system. 


The primary disadvantages of the current system are: 


@ High Cost of Operation 
@ Poor Service Provided by NCTS 


@ 175 Reserve Sites of 354 Total Not Connected to RESFMS 


This analysis will determine if costs of operation are less for the redesigned system. 
Service will no longer be provided by NCTS, but taken in house to COMNAVRESFOR. 
With more direct attention and close control, which are not available when dealing with 
a separate agency, service and support should be better in the redesigned system. 
Finally, the redesign includes a new user interface that will be designed to allow 


connection to RESFMS by all 354 Reserve sites. Since benefits of the redesigned system 
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should be equal or greater than the current system, a straight comparison of costs should 


be sufficient for the redesign decision. 


C. PROJECT LIFE 

As stated earlier, economic life is that period over which savings and benefits are 
expected to accrue; and the economic life chosen for analysis of alternatives is usually 
the shortest of technological, mission, or physical life (Haga and Lang, 1990). 
Sometimes it is necessary to make investments several years prior to the beginning of the 
economic life, which is when the project begins producing savings or benefits. The 
period between initial investment and beginning of economic life is called "lead time" 
(Haga and Lang, 1990). Project life is the combination of lead time and economic life. 

In the case of RESFMS, if new AT&T 3B2/GR computers are purchased they will 
have a physical life of eight to ten years. If the Unisys 1100/92 is retained, it will 
probably have a similar physical life because the system was only recently upgraded from 
an 1100/90 to the 1100/92. Mission life is indefinite, since the function being performed 
is essential to Naval Reserve operations and will continue to be a need for the foreseeable 
future. Technological life is more difficult to determine. Even though available 
technology may quickly and dramatically improve, both the 3B2/GR and the Unisys 
1100/92 are adequate for the task at present. The question becomes one of 
maintainability and cost-effectiveness of technological replacement. NCTS claims that 
they can provide effective maintenance indefinitely for the Unisys 1100/92 or any chosen 


NCTS operated follow-on computer. AT&T will provide full maintenance support for 
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the 3B2/GR for five years. A number of other contractors are available to provide 
maintenance after five years. Therefore, a ten year hardware system life is feasible. 

RESFMS has already been operational for nine years. Investment in software 
development began two years before the first module was operational making RESFMS 
project life, to date, eleven years. The Director of Defense Information has stated that 
one goal of CIM is to achieve 20 year and greater system life for DoD information 
systems. Therefore, an additional ten years of project life for RESFMS in its present 
configuration 1s also feasible. 

Estimates of schedule for software development of the redesign alternative vary 
from 20 to 29.8 months. SASET has been more thoroughly tested and calibrated for 
development projects in the Department of Defense and is, therefore, probably more 
reliable (Gross, 5 May 1992). The schedule estimates from SASET ranged from 20 to 
25 months. It would be reasonable to assume that, if ample resources are applied and 
an optimum schedule pursued, software development for the redesign would probably 
require approximately two years. If however, sufficient funding is not initially made 
available or the high development man-month estimates prove correct, a three year lead 
time may be required. This means that, for this economic analysis, there will be a lead 
time of two or three years before a new system will begin generating savings and 
benefits. During this time the current configuration will continue to operate. 

Considering all these factors, I have chosen a project life of ten years due to the 
physical life limit of the 3B2/GR computers. Economic life starts when the system 


begins to generate savings or benefits. Thus, the economic life will be either seven or 
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eight years depending on whether a three or two year lead time is used. Both scenarios 
will be considered and results for each calculated in the analysis of alternatives for 


RESFMS. 
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VII. ANALYSIS 


A. PROJECT LIFE CYCLE 

The project life cycle will begin with the purchase of at least one AT&T 3B2/GR 
computer. In order to provide adequate computer resources for _ testing 
COMNAVRESFOR desires to have two 3B2/GR computers dedicated to the development 
effort (Furrey, 14 May 1992). Since it will be necessary to test software on the target 
computer platform from the earliest stages of development, the purchase and installation 
of two 3B2/GR computers should take place at the beginning of YEAR ONE of project 
life. Development should take two or three years, during which software development 
costs and 3B2/GR technical support will be needed. The current configuration of 
RESFMS will continue to operate during this lead time. However, costs of operating the 
current configuration will not be considered during this lead time because these costs will 
be incurred regardless of which alternative is selected. The first year in which a system 
begins generating benefits is the beginning of economic life. Economic life will begin 
in YEAR THREE of project life, if a two year lead time is used, or YEAR FOUR, if 
a three year lead time is used. Therefore, the economic life for comparison will be from 
YEAR THREE to YEAR TEN of project life in the case of two year lead time, and from 
YEAR FOUR to YEAR TEN of project life in the case of three year lead time. Salvage 
value will not be considered in this analysis as it is anticipated that technological 


advances in the next ten years will render salvage value of the computer equipment zero. 
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B. DISCOUNTED COSTS 

Department of Defense directives stipulate that costs will first be calculated in 
constant year dollars and that a ten percent discount factor will then be applied to 
calculate the present value of future costs. 

One way to compute the present value of costs is to use a formula and compute the 
exact discount factor for the time the cost is incurred. For instance, if costs occurred 
monthly, the formula could be used to calculate a different discount factor for each 
month. This method is very tedious and time consuming. If costs are evenly distributed 
throughout the year, a similar result can be obtained by simply averaging the discount 
factors for the beginning and the end of year to determine a mid-year factor and applying 
this factor to the total costs for the year. 

A table of ten percent discount factors may be found in Appendix B. These are 
mid-year factors and their use assumes an investment or cost is evenly distributed 
throughout the year. For costs or investments which occur at a particular point in time, 
not distributed throughout a year, a different set of discount factors should be used. 

In the case of RESFMS only the cost of initial purchase of the 3B2/GR computers 
occurs at a single point in time, at the beginning of YEAR ONE. All other costs can be 
assumed to be evenly distributed throughout the year in which they occur. Therefore the 
discount factors found in Appendix B may be applied to all costs in this analysis except 
the cost of initial investment for 3B2/GR computers. Since the purchase of 3B2/GRs 


comes at the beginning of YEAR ONE the present value of that cost will be exactly the 
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Same as the cost itself. In other words, a discount factor of 1.0 is appropriate for this 
initial investment. 

One way to represent the fact that initial investments occur prior to the evenly 
distributed costs incurred in YEAR ONE is to place them in a column labeled YEAR 
ZERO. The discount factor applied then in YEAR ZERO is 1.0 and the factor applied 
in YEAR ONE and subsequent years is the mid-year factor from Appendix B. This is 
the method suggested by Haga and Lang (1991) and the method which will be used in 


this analysis. 


C. PRESENT VALUE 

Present Value analysis was done comparing the Status Quo and Redesign 
alternatives considering both two year and three year lead times. Results of Present 
Value analyses for the two year lead time assumption are summarized in Table XI. 
Results of Present Value analyses for the three year lead time assumption are summarized 
in Table XII. 


Table XI PRESENT VALUE - TWO YEAR 
LEAD TIME 


Net Present Value 


Status Quo $21,9969, 150 
Redesign 7,653,445 
Savings $14,315,705 
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Table Xill PRESENT VALUE - THREE YEAR 
LEAD TIME 


Net Present Value 


Status Quo $18,226,891 
Redesign 6,804,361 
Savings Sa? 7 Sa0 


Tables XIII through XX show the detailed analysis of the net present value of both 
the Status Quo and Redesign alternatives. Each Table showing a ten year cash flow has 
been split into two tables for ease in reading. In this analysis, the expected value for the 


following variables was used and was computed using the type of distribution indicated: 


@ Status Quo Software Maintenance - Triangular Distribution 


NIF Charges to NCTS - Triangular Distribution 
@ Man-Months of Development Effort - Normal Distribution 


@ Redesign Annual Software Maintenance - Normal Distribution 


Values were derived as described in Chapter VI. 
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D. SAVINGS/INVESTMENT RATIO 

The SIR shows the "relationship between future cost savings and the investment 
necessary to obtain those savings" (Haga and Lang, 1991). Savings only accrue during 
the economic life of the project. Investment is normally made during the lead time 
period. In the case of RESFMS, savings begin to accrue in YEAR THREE (two year 
lead time) or YEAR FOUR (three year lead time) and continue to YEAR TEN. In each 
of these years, the current year value of the Redesign Alternative is subtracted from the 
current year value of Status Quo costs to determine annual savings. The appropriate 
discount factor from APPENDIX B is then applied to obtain the present values of annual 
savings. The present value of savings for each year in the economic life are summed to 
form the present value of total savings, PVs. 

In a similar manner, the current year costs of investments are totaled. Then the 
appropriate discount factor is applied to determine present value of each year of 
investment. The sum of all investment years produces the present value of total 


investment, PV,. To determine the SIR, the following formula is used: 


PV, 
PV, 


SIR = 





Tables XXI and XXII detail the computations for PV, and PV); first, in the case 


of two year lead time, and then, for a three year lead time. 
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The resulting values for PV, and PV, are then used to compute the SIR. Thus the 


SIR computation for the two year lead time situation is as follows: 


PVs _ $16,963,397 
PV, $2,647,691 





SIR, = = 6.407 


The SIR computation for the three year lead time situation is as follows: 


E. DISCOUNTED PAYBACK ANALYSIS 

In order to compute discounted payback (PB), the present value of investment and 
an annual savings figure are required. In many cases the annual savings figure is 
constant throughout the economic life. However, in the case of RESFMS, the annual 
Savings figure increases, in constant year dollars, each successive year of economic life. 
Therefore, for this analysis the savings from the first year of economic life will be used, 
since the first year of savings will be the lowest annual amount and thus yield the worst 
case value of the payback factor. 

To determine discounted payback, the present value of investment, PV,, is divided 
by the annual savings in constant year dollars. To this result is added the cumulative 


discount factor from APPENDIX B, Table B for the number of years of lead time. 
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When the final value obtained is compared with APPENDIX B, Table B cumulative 
discount factors, the year in which payback occurs can be determined. 
For RESFMS, payback was calculated for both two and three year lead time. In 


the case of two year lead time, the cumulative factor is computed as follows: 


PV 
I 5 £2, COTS =F 


Annual Savings for YEAR 3 S875) 7,533 


This value is then adjusted for lead time by adding the cumulative factor from 


APPENDIX B for two years. 


O2740 +el.82ia= 2.561 


Comparison with APPENDIX B, Table B shows that 2.561 falls between the YEAR 
TWO cumulative value of 1.821 and the YEAR THREE cumulative value of 2.609. 
This eas that payback occurs within YEAR THREE, or before the end of the first 
year of economic life. 


In the case of three year lead time, the cumulative factor is computed: 


JEN - $2,632,704 _ 9 43¢ 
Annual Savings for YEAR 4 Sep577 333 


This value is then adjust for lead time by adding the cumulative factor from APPENDIX 


B, Table B for three years. 


at 


O° 736 42.002 —- ar 45 


Comparison with APPENDIX B, Table B indicates that 3.345 falls between the YEAR 
FOUR cumulative value of 3.326 and the YEAR FIVE cumulative value of 3.977. Thus 
payback occurs shortly after the beginning of YEAR FOUR, or in the second year of 


economic life. 


F, RISK ANALYSIS 

In order to assess the amount of risk involved in this economic analysis, and to 
determine most probable outcomes, a financial simulation was conducted using @RISK. 
@RISK is a risk analysis and modeling tool, produced by Palisade Corporation, 
Newfield, N.Y.'° It functions as an add-in program to Lotus 1-2-3!!. 

Using a Monte Carlo type simulation, an analysis tool such as @RISK can allow 
decision makers to determine relative interaction between variables, and possible ranges 
of values that may be expected from project execution. Probability of positive outcomes 
may also be determined. 

In the case of RESFMS, four types of values were varied for purposes of the 
simulation. In the Status Quo alternative, Software Maintenance and NIF Charges to 


NCTS were both varied using a Triangular distribution as shown in Table XXIII. 


The particular version of @RISK used for this simulation was the Student Edition 
of @RISK, Version 1.55, distributed by Addison-Wesley Publishing Company, Inc. 


Lotus 1-2-3 is a registered trademark of Lotus Development Corporation. 
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Table XXIII STATUS QUO VARIABLE COSTS 


Variable Cost: High Low Most Probable 
Software Maintenance $1,608,000 $524,124 $1,050,000 
NIF Charges to NCTS $3,514,000 $2,841,000 $3,040,000 


In the Redesign alternative, development effort in Man-Months, and annual maintenance 
FSP were varied using a Normal distribution. An examination of software maintenance 
FSP projections shows that, even though the estimates vary widely between REVIC and 
SASET outputs, the values are always higher in the early years and taper off as project 
life progresses. To insure this relationship was consistently portrayed in the simulation, 
the first year FSP value was made an /ndependent variable and all subsequent years of 
maintenance effort were made Dependent variables. Although all years of maintenance 
effort varied individually, the trend of the Independent variable was used to determine 
the trend of the Dependent variables. For instance, if the value for the first year of 
maintenance, in any single iteration of the simulation, was below the mean, then values 
for subsequent years of maintenance would be chosen from below the mean. This 
assured the proper cost trends would be simulated. Table XXIV below lists variables in 
the redesign alternative and values used for simulation. 

Monte Carlo type simulation was done using 4,000 iterations for the two year lead 
time scenario. Due to computer time constraints, a simulation using 1,000 iterations was 


done for the three year lead time scenario. Results of the simulation were calculated for 
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Table XXIV REDESIGN ALTERNATIVE VARIABLES 
TS SG EE ee ee eee ee eee ee 





Variable Value Mean Standard Deviation 
Man-Months of Development 496.83 135.46 
Year ESk 5.7000 B532902 
Year 2 ESP 4.8450 Ub SSS) 
Year 3 Psr 4.1900 2.73016 
Year 4 FSP 3.6400 2.42592 
eat senor 3.4850 2.58747 
Year OG Bor 3.4475 Oates 
Vee TE Je Se 3.3700 2.70966 
Year 8 FSP 3.2950 2.78980 


EE —=Ee s 
Man-Months of Development Effort, Present Value (PV) of Savings, Savings/Investment 
Ratio (SIR), and Discounted Payback Factor (PB). 

Results of the simulations in the form of histograms and associated statistics are 
displayed on the following pages. The distribution of values and probabilities for Man- 
Months of Development is identical for both two year and three year lead time. The 
resulting cost is simply distributed evenly over the number of years of lead time. 
Therefore, the distribution for Man-Months of Effort is only displayed once, in Figure 
3 and Table XXV. The results for the the two year lead time scenario follow. PV of 
Savings is displayed in Figure 4 and Table XXVI, SIR in Figure 5 and Table XXVII, 
and PB in Figure 6 and Table XXVIII. Then the results of the three year lead time are 
displayed. PV of Savings is displayed in Figure 7 and Table XXIX, SIR in Figure 8 and 


Table XXX, and PB in Figure 9 and Table XXXI. 
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Figure 3 Man-Months of Effort for RESFMS Software Redesign 


Table XXV RISK ANALYSIS STATISTICS FOR MAN- 
MONTHS OF EFFORT 


Expected/Mean Result = 496.8263 
Maximum Result = 968.7614 
Minimum Result = 1.12505 
Range of Possible Results = 967.6363 
Standard Deviation = 135.4097 
Iterations = 4000 
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Present Value of Savings -- Two Year Lead Time 


| L Expected Result= 14.31566 
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Figure 4 Present Value of Savings - Two Year Lead Time 


Table XX VIRISK ANALYSIS STATISTICS FOR PRESENT VALUE 
OF SAVINGS -- TWO YEAR LEAD TIME 


Expected/Mean Result = 14,315,660 
Maximum Result = 19,968,080 
Minimum Result = 8,915,915 
Range of Possible Results = 11,052,170 
Probability of Positive Result = 100% 
Probability of Negative Result = 0% 
Standard Deviation = 1,637,532 
Iterations = 4,000 
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Savings/Investment Ratio - Two Year Lead Time 


| f Expected Result= 6.797859 
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Figure 5 Savings/Investment Ratio - Two Year Lead Time 


Table XXVII RISK ANALYSIS STATISTICS FOR 
SAVINGS/INVESTMENT RATIO -- TWO YEAR LEAD TIME 


Expected/Mean Result = 6.797859 
Maximum Result = 39.27233 
Minimum Result = 3.214479 
Range of Possible Results = 36.05785 
Probability of Positive Result = 100% 
Probability of Negative Result = 0% 
Standard Deviation = 2.008756 
Iterations = 4,000 
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Payback Factor -- Two Year Lead Time 
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Figure 6 Discounted Payback Factor - Two Year Lead Time 


Table XXVIII RISK ANALYSIS STATISTICS FOR DISCOUNTED 
PAYBACK FACTOR -- TWO YEAR LEAD TIME 


Expected/Mean Result = 2.567812 
Maximum Result = 3.318428 
Minimum Result = 1.941221 
Range of Possible Results = 1.377207 
Probability of Positive Result = 100% 
Probability of Negative Result = 0% 
Standard Deviation = 0.1837197 
Iterations = 4,000 
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Present Value of Savings -- Three Year Lead Time 


P 
R 
O 
B 
A 
B 
| 
L 
T 
Y 


8.5 9.75 11 12.25 13.5 14.75 16 
Values in Millions 





Figure 7 Present Value of Savings - Three Year Lead Time 


Table XXIX RISK ANALYSIS STATISTICS FOR PRESENT VALUE 


OF SAVINGS -- THREE YEAR LEAD TIME 
— iS. Se ccil 


Expected/Mean Result = 11,422,680 
Maximum Result = 15,543,950 
Minimum Result = 6,527,336 
Range of Possible Results = 9,016,610 
Probability of Positive Result = 100% 
Probability of Negative Result = 0% 
Standard Deviation = 1,407,193 
Iterations = 1,000 
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Savings/Investment Ratio - Three Year Lead Time 
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Figure 8 Savings/Investment Ratio - Three Year Lead Time 


Table XXX RISK ANALYSIS STATISTICS FOR 
SAVINGS/INVESTMENT RATIO -- THREE YEAR LEAD TIME 


Expected/Mean Result = 5.63609 
Maximum Result = 19.34061 
Minimum Result = 2.862788 
Range of Possible Results = 16.47782 
Probability of Positive Result = 100% 
Probability of Negative Result = 0% 
Standard Deviation = 1.567336 
Iterations = 1,000 
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Payback Factor -- Three Year Lead Time 


7] Expected Result= 3.352199 
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Figure 9 Discounted Payback Factor - Three Year Lead Time 


Table XXXI RISK ANALYSIS STATISTICS FOR DISCOUNTED 
PAYBACK FACTOR -- THREE YEAR LEAD TIME 


Expected/Mean Result = 3.352199 
Maximum Result = 4.01652 
Minimum Result = 2.808435 
Range of Possible Results = 1.208085 
Probability of Positive Result = 100% 
Probability of Negative Result = 0% 
Standard Deviation = 0.1789016 
Iterations = 1,000 
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G. FINDINGS 


1. Comparison of results 

In the economic analysis using expected values of variables, all three 
economic analysis tools used produced results favoring the redesign alternative. Using 
Net Present Value analysis, present value of the savings were positive. The three year 
lead time yielded the lower value, which was still in excess of $11 million over the life 
of the project. 

A SIR greater than 1.0 should be obtained before a project should be 
considered economically sound (Haga and Lang, 1991). In the case of RESFMS, 
expected value computations indicate a SIR of 5.339 for the three year lead time 
alternative, and 6.407 if development is completed in two years. 

Discounted payback analysis shows that payback is expected to occur within 
the first two years of economic life. For the two year lead time scenario, payback was 
within the first year of economic life. In the three year scenario, payback is expected 
to occur soon after the beginning of the second year of economic life. This is a short 


payback period and indicates a strong economic investment. 


2. Risk Analysis 
Examination of histograms and related statistics for financial simulation 
performed for the RESFMS project supports the positive results of expected value 


computations discussed above. 
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In simulation of both two and three year lead time scenarios, PV of savings 
remained positive for all iterations of the simulation. Probability of a positive result was 
100% for both scenarios. The minimum result obtained by simulation was $6,527,336, 
calculated in the three year lead time scenario. Examining the histograms, we find the 
cumulative probability for a PV of savings value greater than $11 million is more than 
90% for the two year lead time and about 60% for the three year lead time. Therefore, 
regardless of lead time scenario chosen, PV of savings can be expected to be no less than 
$6.5 million and there is more than a 60% percent chance that it will be greater than $11 
million. 

Worst case for SIR in the simulations still shows a ratio of 2.86 in the three 
year lead time scenario. Probabilities indicate that a value above 5.0 is much more 
likely. Thus, even in the worst case, SIR would indicate the redesign alternative is an 
economically sound project to pursue. 

Simulation results for discounted payback analysis show that payback will 
probably occur within two years of project implementation regardless of whether lead 
time is two or three years. The worst case found in the simulations was a value of 4.016 
in the three year lead time scenario. Even this very low probability result would have 
payback occur within the third year after the beginning of economic life. Since economic 
life is expected to last seven years in the three year lead time scenario, savings beyond 
the value of initial investment would still accrue for at least three years more. 

Therefore, risk analysis shows no probability of negative values, given the 


range of values and assumptions made in this analysis. I have endeavored to accurately 
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portray ranges of possible values, to include all possible costs, and to completely quantify 
significant economic risk. Asa result, I believe the risk analysis simulations performed 


are a reliable indicator of potential outcomes. 


3. Patterns Discerned 
All results obtained were positive. There were no negative results. Even the 
worst case results of simulations show PV of savings, SIR, and payback periods well 
within acceptable ranges for project approval. All these results affirm the economic 
prudence of pursuing the redesign of RESFMS. Baring unforseen technical problems that 
significantly alter development effort estimates, RESFMS redesign should be 


economically successful. 
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VII. RECOMMENDATION 


A. RESULTS OF ECONOMIC ANALYSIS 

The results of all economic analyses indicate a favorable economic outcome if the 
RESFMS redesign is implemented. All factors considered in this analysis favor the 
redesign over the status quo. A preliminary technical analysis reveals no significant 
areas of concern, involving hardware, resulting from the move of RESFMS from a 
mainframe environment to a minicomputer. The primary expense, and greatest effort 
required, will come from rewriting software code. In spite of the difficulty encountered 
in deriving an estimate of the cost of software development, I believe that a reasonable 
range of potential values was produced and adequate risk analysis performed to ensure 
that potential costs were reliably represented. 

Since the results of all tools used and of all simulations performed strongly indicate 
the economic soundness of the redesign alternative, I recommend that the redesign of 
RESFMS to operate in the configuration described in this analysis be approved and 
initiated. It is significant to note that greater savings, higher SIR, and shorter payback 
were all predicted for the two year lead time than for the three year lead time scenario. 
Actual schedule duration for the redesign development effort will be determined by the 
level of funding made available for the project. If funding is low and spread out over 
several years, fewer programmers will be assigned and project development will be 


delayed. Although this analysis indicates that positive results will still be achieved, those 
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results will be less than can be obtained if wholehearted approval is given and adequate 


resources are allocated to allow a short development time. 


B. OTHER CONSIDERATIONS 

Even though the redesign of RESFMS would mean significant savings to the Naval 
Reserve, there are two factors which may prevent the development effort from being 
approved. First, monies to operate systems and monies to develop systems do not 
necessarily come from the same appropriation. Also, there are very specific approval 
procedures required for development efforts that do not apply to currently operating 
systems. In particular, the nearly $300,000 required to purchase two 3B2/GR 
minicomputers would necessarily come from the Other Procurement Navy (OPN) 
appropriation and would require special approval before it could be spent (Practical 
Comptrollership, 1992). Also, if the cost of software development is not competitively 
bid and exceeds $250,000, or is competitively bid and exceeds $2.5 million, an Agency 
Procurement Request must be sent to GSA for approval (GSA Overview Guide, January 
1990). These additional requirements and constraints might cause approval to be 
delayed, or the project to be canceled as the Naval Reserve competes with other DoD 
agencies for Information System development and acquisition funds. COMNAVRESFOR 
personnel must effectively communicate the significant savings that a redesigned 
RESFMS can produce and as a result obtain funding and approval required to continue. 

The second consideration is that, although moving RESFMS to minicomputers 


controlled by COMNAVRESFOR may produce great savings for the Naval Reserve, it 
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may not necessarily save the Department of the Navy anything at all. In fact, depending 
on what NCTS does in response, it may actually cost DoN more than the status quo. 
This is due to the fact that NCTS is a Navy owned and operated organization. Thus, 
NCTS funding comes from DoN budgets just aa COMNAVRESFOR funding does. If 
NCTS is charging COMNAVRESFOR based on actual costs of maintaining their facility, 
and if no replacement customers are found when COMNAVRESFOR moves, these 
overhead costs may still be incurred by the Navy as a whole but no longer charged to the 
Naval Reserve. Of course, a reasonable person would conclude that if NCTS is not 
providing cost effective service then the Navy should reconsider its support. In any case, 
the actions of NCTS are beyond the scope of this analysis. What is clear from this 
analysis is that the Naval Reserve can save significant amounts of future operating 
expense if RESFMS is redesigned. If all Navy activities pursued projects that resulted 
in operating expense savings, the Navy as a whole would have to benefit. Therefore, the 
redesign of RESFMS should be evaluated on its own merits and a development decision 


should not be affected by short term effects on other service agencies. 


C. CONCLUSION 

This economic analysis indicates that a clear economic advantage to the Naval 
Reserve will result from a redesign of RESFMS. It logically follows that the Department 
of the Navy will ultimately benefit from reduced cost in funding the Naval Reserve. To 


gain maximum savings, adequate resources should be allocated to develop the new 
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system in minimum time. Therefore, the Naval Reserve should pursue a redevelopment 


effort and DoN fully support the effort to allow early realization of projected savings. 
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ADSR 

ADPE 

ADT 

AIS 

AT 

BCR 

BE 

bps 

CASE 

SERPS 
CINCLANTFLEET 
COMNAVRESFOR 
CNRF 

CNRF Code 02 


CNRFE Code 06 


CNRF Code 10 


APPENDIX A 


ACRONYMNS AND ABBREVIATIONS 


Automated Data Service Request 

Automated Data Processing Equipment 

Active Duty Training 

Automated Information System 

Annual Training 

Benefit/Cost Ratio 

Break Even 

bits per second 

Computer Aided Software Engineering 

Centralized Expenditure Register Processing System 
Commander in Chief, U.S. Atlantic Fleet 

Commander Naval Reserve Force 

Commander Naval Reserve Force 

Commander Naval Reserve Force Director of Finance 
Commander Naval Reserve Force Director of Manpower 
and Personnel 

Commander Naval Reserve Force Director of Information 


Systems 
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CRU 


DBOF 


DDI 


DDN 


DoD 


DODD 


DODI 


DoN 


DI 


DSI 


ne 


FIP 


FIRE 


FIRMR 


FM 


EP 


ole 


FY 


GAO 


GB 


GC 


Central Processing Unit 

Defense Business Operating Fund 

Director of Defense Information 

Defense Data Network 

Department of Defense 

Department of Defense Directive (same as Instruction) 
Department of Defense Instruction (same as Directive) 
Department of the Navy 

Degree of Influence 

Delivered Source Instructions 

Unadjusted Function Point Count 

Federal Information Processing 

Financial Information Processing Center 

Federal Information Resources Management Regulation 
(published by U.S. General Services Administration) 
Financial Management 

Function Point(s) 

Full-time Software Person 

Fiscal Year 

General Accounting Office 

Gigabytes 


General Characteristics 
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GCA 


GTS 


GSA 


CoUss 


HASC 


Ese IC 


IDTT 


IFPUG 


IMAPMIS 


I/O 


IRM 


IRMC 


IRR 


Te 


KDSI 


KLOC 


LCM 


LAN 


General Characteristics Adjustment 

Government Travel System 

U.S. General Services Administration 

Government Transportation Request 

U.S. House of Representatives Committee on Armed 
Services (House Armed Services Committee) 

Health Science Education Training Command 

Inactive Duty Training Travel 

International Function Point User Group 

Inactive Manpower and _ Personnel Management 
Information System 

Input/Output 

Information Resources 

Information Resources Management 

Information Resources Management College (within 
National Defense University) 

Individual Ready Reserve 

Information Technology 

Thousands of Delivered Source Instructions 

Thousands of Lines of Code 

Life-Cycle Management 


Local Area Network 
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LOC 
MAC 

MB 

Mbps 
MCPS 
MIPS 
MM 

MNS 
MPT 
NARDAC 
NAVPTO 


NCTS 


NIF 
NRPC 
OMB 
OPTAR 
PM 
Pak 
PB 

PV 


R&D 


Lines of Code 

Military Airlift Command 

Megabytes 

Megabits per second 

Microcomputer Claims Processing System 
Millions of Instructions Per Second 
Man-month (of effort) 

Mission Needs Statement 

Manpower Personnel and Training 

Navy Regional Data Automation Center (now called NCTS) 
Navy Passenger Transportation Office 
Navy Computer and Telecommunication Station (formerly 
NARDAC) 

Navy Industrial Fund 

Naval Reserve Personnel Center 

Office of Management and Budget 
Operating Target (for budget expeditures) 
Program Manager 

Problem Tracking System Report 
Discounted Payback Period 

Net Present Value Analysis 


Research and Development 
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RAM 


RESCOMMIS 


RISC 


RTS 


RHS 


RPN 


RSTARS 


RTSS 


SASET 


SATO 


SELRES 


Biel 


SEMA - 


SIR 


SPA 


meP/ TP 


UAC 


UCA 


Random Access Memory 

Reserve Command Management Information Strategy 
Reduced Instruction Set Computing 

Request for Transportation Services 

Reserve Headquarters System 

Reserve Personnel Navy (Congressional appropriation) 
Reserve Standard Training Administration and Readiness 
Support 

Reserve Training Support System 

Software Architecture, Sizing, and Estimating Tool 
Scheduled Airline Ticket Office 

Selected Reservists 

Software Engineering Institue, Carnegie Mellon University 
Systems Engineering and Management Associates, Inc. 
Savings/Investment Ratio 

Software Process Assessment (performed by SEI) 
Transmit Control Protocol/Internet Protocol 

Uniform Annual Cost 


Uniform Chart of Accounts 
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APPENDIX B 


PROJECT YEAR DISCOUNT FACTORS 





Year Table A Table B 
PRESENT VALUE OF $1 (Single PRESENT VALUE OF $1 
Amount used when cash _ flows (Cumulative Uniform Series to be 
accrue in varying amounts each used when cash flows accrue in the 
year). same amount each year). 

1 0.954 0.954 

Z 0.867 1.821 

3 0.788 2.609 

+ 0.717 3.326 

5 0.652 Se 

6 0592 4.570 

l 0.538 5.108 

8 0.489 5.507 

9 0.445 6.042 

10 0.405 6.447 

11 0.368 6.815 

l2 0.334 7.149 

13 0.304 7.453 

14 0.276 T9729 

15 0.251 7.980 

16 0.228 8.209 

17 0.208 8.416 

18 0.189 8.605 

19 0.172 Bed 

20 0.156 8.933 

21 0.142 9.074 

22 0.129 9.203 

23 0.117 0320 

24 0.107 9.427 

Bae 0.097 9.524 


NOTE: Table B factors represent the cumulative sum of Table A factors through any given project year. 


144 


LIST OF REFERENCES 


Abdelsamad, Moustafa H., Evaluating Capital Expenditure Proposals in Large Industrial 
Corporations, D.B.A. Dissertation, George Washington University, June, 1970. 


Albrecht, Allan J., "Measuring Application Development Productivity," Proceedings of 
IBM Application Development Symposium, pp. 83-92, Monterey, California, October 
1979. 


Albrecht, Allan J., and Gaffney, John E. Jr., "Software Function, Source Lines of Code, 
and Development Effort Prediction: A Software Science Validation," JEEE Transactions 
on Software Engineering, v. SE-9, no. 6, pp. 639-648, November 1983. 


Albrecht, AllanJ., AD/M Productivity Measurement and Estimate Validation, IBM Corp. 
CIS & A Guideline 313, IBM Corporate Information Systems and Administration, 1 
November 1984. 


Telephone conversations between Tom Albro, Code 102A, Commander Naval Reserve 
Force and the author, 7 April, 20 May, and 13 July 1992. 


Barbetta, F., "Burroughs Wins Sperry in $4.7B Takeover", Electronic News, V. 32, p. 
1, 2 June 1986. 


Behrens, Charles A., "Measuring the Productivity of Computer Systems Development 
Activities with Function Points," /EEE Transactions on Software Engineering, v. SE-9, 
no. 6, pp. 648-652, November 1983. 

Brewin, Bob, "CIM," Federal Computer Week White Paper, p. 3, September 1991. 
Blaylock, Jerome W., "An Expert Analysis of the Reserve Financial Management 
System," draft of technical report prepared for Commander, Naval Reserve Force, New 


Orleans, Louisiana, May 1990. 


Boehm, Barry W., Software Engineering Economics, pp. 35-54, 98-160, 329-343, 492- 
531, Prentice-Hall, Inc., 1981. 


Cash, James I., and others, Corporate Information Systems Management, 2d ed., p. 1, 
Richard D. Irwin, Inc., 1988. 


145 


Interview with Mr. Ron Chamberlain, Budget Director, Commander Naval Reserve 
Force Code 101, New Orleans, Louisiana, by the author, 13 May 1992. 


Commander Naval Reserve Force Code 101, RESFMS Ad Hoc Budget Expenditure 
Report for FY90-FY91, 14 May 1992. 


Commander Naval Reserve Force Code 101, RESFMS Ad Hoc Budget Estimate Report 
for FY92-FY97, 14 May 1992. 


Commander Naval Reserve Force Code 105, RESFMS Redesign Briefing Paper, August 
1991. 


Commander Naval Reserve Force Code 105, RESFMS Briefing Notes, June 1991. 


Interview with Mr. William Curtis, Software Engineering Institute, Carnegie Mellon 
University, Pittsburgh, Pennsylvania, by the author 4 May 1992. 


Department of Defense Instruction (DODI) 7041.3, "Economic Analysis and Program 
Evaluation for Resource Management", 18 October 1972. 


Department of Defense Instruction (DODI) 7920.2, "Automated Information System 
(AIS) Life-Cycle Management Review and Milestone Approval Procedures," 7 March 
1990. 


Director of Defense Information (DDI), Office of the Assistant Secretary of Defense, 
Memorandum, "Corporate Information Management (CIM) Business Case (Functional 


Economic Analysis)," 23 July 1991. 


Endoso, Joyce, "Ire Over CIM Spells Budget Cuts," Government Computer News, v. 11 
no. 14, p. 1, 6 July 1992. 


Interview with CAPT P. D. Faubel, Comptroller, Commander in Chief U.S. Atlantic 
Fleet, by the author 19 May 1992. 


Interview with LCDR William Furrey, Project Manager for RESFMS Redesign, 
Commander Naval Reserve Force Code 105C, New Orleans, Louisiana, by the author, 


14 May 1992. 


Telephone conversation between LCDR William Furrey, Code 105C, Commander Naval 
Reserve Force and the author, 7 June 1992. 


Garrison, Ray H., Managerial Accounting, Richard D.Irwin, Inc., 1988. 


146 


Gross, Stephen, "Software Cost Estimation," presentation to the Naval Postgraduate 
School, Monterey, California, 5 May 19972. 


Haga, William J., and Lang, Robert G., Economic Analysis Procedures for ADP, 
Revised ed., Naval Postgraduate School, June 1991. 


Hihn, Jairus, and Habib-agahi, Hamid, "Cost Estimation of Software Intensive Projects: 
A Survey of Current Practices," 199] IEEE 13th International Conference on Software 
Engineering, pp. 276-287, IEEE Computer Society Press, 1991. 


Jones, Capers, Programming Productivity, McGraw-Hill Book Company, 1986. 


Karpinski, R., "AT&T Uses Meeting To Tout NCR Deal", Telephony, v. 220, p. 9, 22 
April 1991. 


Kellner, Mark, "Data Power," Government Executive, pp.36-47, August, 1991. 


Kile, Raymond, REVIC Software Cost Estimating Model User’s Manual, Version 9.0, 
Naval Center for Cost Analysis, 9 February 1991. 


Lacy, Coreen, COMNAVRESFOR POM 94 Briefing Paper, Subject: RESFMS Upgrade, 
12 August 1991. 


Interview with Coreen Lacy, RESFMS Program Manager, Commander Naval Reserve 
Force Code 105A, New Orleans, Louisiana, by the author, 30 June 1992. 


Lang, Hans J., Cost Analysis for Capital Investment Decisions, p. 108-109, Marcel 
Dekker, Inc., 1989. 


Telephone conversation between Ben Lazar, Head Analyst, RESFMS Software 
Maintenance Group, SEMA Corp., New Orleans, Louisiana, and the author 16 July 
1992. 


Levy, Haim and Sarnat, Marshall, Capital Investment and Financial Decisions, 2d ed., 
p. 26, Prentice-Hall International, Inc., 1982. 


Lorin, Harold, Aspects of Distributed Computer Systems, 2d ed., p. 262, John Wiley & 
Sons, 1988. 


Mendelsohn, M., and others, The Student Edition of @RISK User’s Manual, p. R-90, 
Addison-Wesley Publishing Company, Inc., 1991. 


147 


NCR Federal Systems Division Letter to the author, Subject: Requested 3B2 Pricing, 
22 ull too 


Norden, P., "Useful Tools for Project Management," Software Cost Estimating and Life 
Cycle Control, pp. 216-225, [EEE Computer Society Press, 1980. 


Practical Comptrollership, Revised Edition, Module A, p.25-26, Naval Postgraduate 
School, Monterey, California, January 1992. 


Pressman, Roger S., Software Engineering, A Practitioner’s Approach, 3rd ed., pp. 45- 
56,315-365, McGraw-Hill, Inc., 1992. 


Quirin, David G and Wiginton, John C., Analyzing Capital Expenditures, p. 25, Richard 
D. Irwin, Inc., 1981. 


Interview with CAPT Ron Rautenberg, Deputy Director Information Systems, 
Commander Naval Reserve Force, by the author, 15 September 1991. 


Royce, W. W., "Managing the Development of Large Software Sy6stems: Concepts and 
Techniques,}" Proceedings, WESCON, August 1970. 


Silver, A., and others, SASET User’s Guide, Prepared for Naval Center For Cost 
Analysis, Martin Marietta Denver Aerospace Corporation, February 1990. 


Interview with Mr. David A. Spivey, Program Manager, Information systems 
Modernization, U.S. Army Corps of Engineers, Washington, DC, by the author 4 June 
1992. 


Stermole, Franklin J., Economic Evaluation and Investment Decision Methods, 5th ed., 
pp. 156-157, Investment Evaluations Corp., 1984. 


Stevens, G.T. Jr., Economic and Financial Analysis of Capital Investments, p. 2, John 
Wiley & Sons, 1979. 


Strassmann, Paul A., [Information Payoff, pp. 152, 270, The Free Press, 1985. 


Strassmann, Paul A., The Business Value of Computers, pp. 195-224, The Information 
Economics Press, 1990. 


Interviews with Myung W. Suh, Department of Administrative Sciences, Naval 


Postgraduate School, Monterey, California, by the author, October 1991, and 9 July 
ee 


148 


U.S. House of Representatives, Committee on Armed Services (HASC), Report on H.R. 
2461, National Defense Authorization Act for Fiscal Years 1990-1991, Government 
Printing Office, Washington, DC, 1 July 1989. 


U.S. House of Representatives, Appropriations Committee (HAC), Report on Department 
of Defense Appropriations Bill 1990, Government Printing Office, Washington, DC, 1 
August 1989. 


U.S. House of Representatives, Committee on Government Operations (HGOC), Sixth 
Report, DoD Automated Information Systems Experience Runaway Costs and Years of 
Schedule Delays While Providing Little Capability, Government Printing Office, 
Washington, DC, 20 November 1989. 


U.S. General Services Administration, Office of Information Resources Mangement, 
Federal Information Resources Management Regulation (FIRMR), Chapter 201, Subtitle 
E--Federal Information Resources Management Regulations System, Title 41--Public 
Contracts and Property Management, Code of Federal Regulations, Amendment 1, pp. 
20-3 to 20-4, Washington, DC, October 1990. 


U.S. General Services Administration, Information Resources Management Service, 
Overview Guide for the Acquisition of Information Resources, p. 3-1, Washington, DC, 


January 1990. 


U.S. Office of Management and Budget, Circular No. A-94, Discount Rates to be Used 
in Evaluating Time-Distributed Costs and Benefits, Washington, DC, 27 March 1972. 


Walker, Charles G., Economic Analysis of Information Systems, Master’s Thesis, Naval 
Postgraduate School, Monterey, California, March 1991. 


Zipper, S., "AT&T Would Drop Own CPU Units if NCR Bought", Electronic News, 
fmo0, p. 1, 10 December 1990. 


Interview with Dani Zwieg, Department of Administrative Sciences, Naval Postgraduate 
School, Monterey, California, by the author, 9 May 1992. 


149 


INITIAL DISTRIBUTION LIST 


Defense Technical Information Center ..................——e 
Cameron Station 
Alexandria, Virginia 22304-6145 


Library, Code 0142 . . 2.9%... | eg eee 
Naval Postgraduate School 
Monterey, California 93943-5002 


Commander Naval Reserve Force, Code 10 ....... . 0") ee 
13000 Chef Menteur Highway 
New Orleans, Louisiana 70129-1800 


Dr. William J. Haga, Code AS/Hg .........:.....+ 45 
Naval Postgraduate School 
Monterey, California 93943-5000 


Dr. Dan Cx Boger; Code AS/BO” 222 me = = ee 


Naval Postgraduate School 
Monterey, California 93943-5000 


150 








HMUOLEY Views eT 
NAVAL POSTGRADUATE SCHOOL 


MONTEREY. 


-5002 


CALIFORNIA 93943 





Lt 
ee erry i tad 
Bipsovemnes “nesters 


eb 
Per net 


Cavin ete Oa 


vd ee rie 


Fabs fi are 


4 a 
oat gine 
Real retet ae OP, 


Het into! 
ry 


¢ Sieh ait 


Wi ta oie, 9 


4 t 
he faqs Hs 


megegr yes oh 
avatar ek pda mr at ' 
. &. ih 


3? $on 
Pe 


Ag ge <E ae, dabted (Pw 


5 ay 
a aie Lie epeegt apres Pea" * 


a sey sia Dep 
rag 


ered Ca gig? 5 
pA sa a 
FaM nS oct te niyo 


iy 
i On, areee 
ere oF ute? ye hd Ser 


rows, 


i 
eere™ ini ed a 
mie Li 
yas Ce 
Fa Pat my were ened 


otgaee 


om 


eure 


erat te # 


3 
er? * 


Pane pre yee tet et if 


“eter 
sa 
Sogetenerr 

oe pee, 


a 


he 
rari bre 8" 


thas He aU Erte ad, 


rises ert 
ae 


SS Caeiitag 


Sarit 
ate ra aed 


aes tite eect: retary habety " dyres 
tierinds oa 


ts yO faker ad FSB SF Sone oF | 
pasicte bArty Pa mapinta Thue aes 

ohaters 5! 

mina t wer tcaet 
parr Ce * 
ha 


otal Ut BEAN Ame 
peice ue ae apases 


eel 4s, 
pane sere 
ny hat 
or «te 
baiataee senrece sim 


OTT Ly eet i 
sae 3 Fs ren Ms =h . : Tete 


a 
wi Bath te, aed te a;Ue6s iS rs cr 


1) 

et Apel 
shel teiahel a Aeeae 
cep be he isis 


fact, nthe oie 
le AT He 
ate Rta a id od ae ihe 


2 ifs 
ry ae é 


re ae vies 
We 2tity he 
eres 
- iia Pe 
Chiles 1s M8 FOOL TDL ET tofmDi Te 
Reyne Mag arse e hed 
avi grata tia 13h Oe DF 


wl Vlad 
oT) a Lo hod 
igedea le Leesa 
wetae eae ta 
we pt ate Baba heh 
Mate brie pfs 
ae yf Sa 
Pata atetate 
pith, erache 
baa ig ns 3 at 
rata 


raabesataeets wolsaa: 
orb el eT ‘ 
PSS Moray 
qe & i 
Cree 
ew 
ewe. tees Sr erinet 
186 fe rede’ Hot 


»" 
atl C addi we 
Petar tt 

BA 


Li spiee ni 


A + np hesarmeet 
ceed phe ecogt 


Esa ick 
Pid 7 ie 
tintabe?® vite i 


Mute Ae 


Ls 1 
ath Rate Cigna. oh: 


ha Bhi a bares Be at hy 
bal ars 


te 
pee feetpiibedait 
tase re 


aut 
& what amarial a 
Reradeel 


ae teu, 

pth peck chy “tifa! nar, 
A od # arent ats Rais hg! 
3 cae seybe EAE 


bn 
ane KY 
f) fp ban f age 
Habe tyel 9 tae OM OE Ye oes 


te FE 
a te ssi 


Konet he 


a deghp eed bite, asl 
at i Lugoar 
5 bs 


sive! Peed Pe gist yt 
“eh a « 
Renae dit gba stats 


eye 
Fags tn 
+ 


ade 
he eatega hs ha Sy 
; igh. iyceren 
ta” weds phe ltt 
scaeecs atest. 


coda: Pera “oe 
fewer 
vine MG Para it te 


Baha, ; 
ata | ee 


ai 


ogg G's 
sat wists 
2 af pipt 


Praha 
H ji 
val . 
Depitiorass 
Maleate Fy! uj 
Sea ewibe & 
ve Mal 


oP et 
“ isd 5 


shy 


Ae ae 
iw 


xa 
= 


eo sty, wqaed 
ne Pe . 


72 


ee” 
sPytalas 
ied | 

i ff ow 

ny 

eh aper 

HG Ue ¢ 

oop yt rer} 


wt oe ae 
ran 
20 at yer 


we 


eds 
Vs ad f a5 


Pirie eg 
Fever unin ti 
iim 7, 


ae awe, aut 
a pee 
a LT Re 5 


Ot Srenteahh RY 
tat 


ip? eres owes eytgt 
im rte ty Cort CF 
Y, 


sty g pit 5 


yee! gel 
“4 Le eee 
2 ‘aby Ee > 


ba 

arial! vy frist qe! a 
+ 

‘ aft 


Guts sis , 
i, oe Ge eae thm rou La at, 
Lets ats fel 


fo 
4 zt 
fey y 


2 Hae 4 eae 
he s pe hanetiee 
Phd bal kel fale ter A 

Cats Gay et rt fe 
Be Se : ae tte dadyter yt Ay 
at bese Ke Hos fi yee Fabs ofl ie 


ma 
aa 


ty ati 
+ 


‘ Ah 
me aly 
f rh T Ey! Het 
bee fe” ah eure Mee Fy ee ved 
gy igi Bi ya 


oy ve gti forge 


wy ett 


reared ® 


cig 


wey 
Seton 


: 
er i fi rapist 


bypteSs 


Talat * ay ewe? 


He ey 
PGs We Of kl ga SES SD 
04 Py™ aimee hy Pod 
dare tehen oH res 
ptiut, € ni \ ” 
cresyl gee Wen 
of? ae FS at pe 
greene 


Ve th bite 


yratel 
ae GM 


ey ne 


wet 
tae Rite 

att it, 
tngeertetgs Po 

gre te 

Seer 

te uj 
ater et é 
template 


ier, 
ihe he rt 


aoe 
ory fod 


jafutee terama’ A Put wee 


a al oo 
aan 


df. 
sient ia ry, it 
Jshg AS Mn 
Hevty hea 
bead sae 
? “yh 


a pty 


cs 
33 vets kee at 
ase 8 ealataeaig 
H 


« i: 
th inca +) pare 
Wt alas 


« 
ee) 


fds DMs. oe 
edhe an wipe we 
epee te 

" 


wi vethiy Gide gy 
RATE ay 


sunirey 
oo ayaa tite st 
ity 


eee areal 
4 


te “yt 


tor aft 

Oa eta 
pt Fe fi ® 
tee 


he wig gat rales 


on PIO 


4 
ret. ‘sear ji 


eens eae 

ath # 
oh tre gh her she 
tos Wh 


af fy 


en Pe 
treisets 
om pe tt FQ: 


hee 45) 
cri range, grersts 
Fhe Fa He has 


aged Be 
Prot Ly 


BP Wis ty die? 
elalye lp ase 


"ote, 


Heel b 


Neat 
i) tie! ae 


ire ohn 
+4 caret 


LY ds tod toe: ab 
é i] 
Peas reed oe iH 
wae ' 
Slab at 
She feo Bs 
geet” 


{ tw ehna gee 
ard A me 


eee 


gf " 
! sat dy ara sae 


te 


tbh (1 1g 
- 'E ET Peet aat=3, 
“ay Hai 4 Piety 


ony 7S Dyb Ray 
MG ar hte 


the 


"t 
matent nin bed am * ty” a ‘ pCa m ler z ; 
are age 


CEN betes) epee s Wy astert — 
re MEO, etal SE , i a : ’ cid ose } 


f= we? Gad 


dgree Ueatt 
eed as ask Orne 


oth te FF 


‘ne only 
rt Vet} 
we hon.tir, Si i 
try Get ae ate is 4 
fa st Vax afre 
teenage 


iter 
ad Lt 
eit 


5 yt 8 
rl ta Py fe" 4h 
sine ase gedt 


474% 8s 
tf tae. 
re Sf pein re 
at helt qin yon te +f 
ty fot be! 
gout eted * 
Areebeciyt ts 
re r 
zeshadest 
a 
Pada « cam @ thes + 
4, ah & wed 
hfs var ¢ 
>a f 3 
* Pree 
ta aed Ts 
valet Be* Te 


Sa tas Tj 


Tee: 
“tee Fee 


Bae tin & 


wey 
cpeoky 


oP why tabies 
> bptete! 
tapes, 
had ce 
opp tale 


DE, tag Pie of 


Hat phos 


a] 
Bobs that e s 
ae aes) rye bred 
é % 
Rae yetgear oul é a Are 
Bite. t A an ergs ' 
ohgt ef ata ee ph » a? 
eka Ree DOQ UM ea LEE A 
eFeom Bae Uy 


Enotg 
sa 


bee te 

Ce Ui er eee 
seh Se 
ae enh He 
is ee 


ee 2% sate (Ch 
ataretel. ae 2 i ate J ea Cah AS 
: ‘ . aed Be garters 
fens 


taper any 
ifayand 
Hear og a yet 
vahie sty fee 
Le amy be" ao af rhs 


t 
ri 2 vatale heady , ee 


aa 7 


“aver 
egit meee 
tof * “te = 
trey tent 


Dyeen,4, 
sete * at 
afaf 


pty 

PURO owetats 
beat OF Es “evade 
wat ep ty ae “Pee 

ne hd. is’ eeu ye? 2% + 

‘ 4 oF gh, tye 

ae tga teh pet 
12 he a tre fies 4: 


sitet eee OT EDY 
ot ratehele 
s es tots 
sr gears ie halts 
ange 
iekenerceys 


hy ako 
‘ : 
er mae 


ong 
- 


ee 
tex? 
Fase t 
Seta 


pry, . 
RP. Ren 
feat 
ena 
nm 


Mew , 
“4 ue, 





